-- Logs begin at Thu 2022-06-23 11:13:32 UTC, end at Thu 2022-06-23 12:16:38 UTC. -- Jun 23 11:40:29.844590 dv-cinder-826608-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 23 11:40:29.854784 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 23 11:40:29.857429 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 23 11:40:29 2022] *** Jun 23 11:40:29.857429 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 23 11:40:29.857429 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 23 11:40:29.857429 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: nodename: dv-cinder-826608-3-iscsi Jun 23 11:40:29.857429 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: machine: x86_64 Jun 23 11:40:29.857429 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: clock source: unix Jun 23 11:40:29.857429 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: pcre jit disabled Jun 23 11:40:29.857429 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: detected number of CPU cores: 4 Jun 23 11:40:29.857429 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: current working directory: / Jun 23 11:40:29.857429 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: detected binary path: /usr/bin/uwsgi-core Jun 23 11:40:29.857429 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: your processes number limit is 47680 Jun 23 11:40:29.858047 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: your memory page size is 4096 bytes Jun 23 11:40:29.858047 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: detected max file descriptor number: 2048 Jun 23 11:40:29.858047 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: lock engine: pthread robust mutexes Jun 23 11:40:29.858047 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: thunder lock: enabled Jun 23 11:40:29.858047 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 23 11:40:29.858047 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 23 11:40:29.901439 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: Python main interpreter initialized at 0x5586a956b9c0 Jun 23 11:40:29.901439 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: python threads support enabled Jun 23 11:40:29.901439 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: your server socket listen backlog is limited to 100 connections Jun 23 11:40:29.901439 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: your mercy for graceful operations on workers is 90 seconds Jun 23 11:40:29.901771 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: mapped 403077 bytes (393 KB) for 2 cores Jun 23 11:40:29.901898 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: *** Operational MODE: preforking *** Jun 23 11:40:29.902125 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: *** uWSGI is running in multiple interpreter mode *** Jun 23 11:40:29.902125 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: spawned uWSGI master process (pid: 80234) Jun 23 11:40:29.902316 dv-cinder-826608-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 23 11:40:29.903407 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: spawned uWSGI worker 1 (pid: 80243, cores: 1) Jun 23 11:40:29.903407 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: spawned uWSGI worker 2 (pid: 80244, cores: 1) Jun 23 11:40:29.903407 dv-cinder-826608-3-iscsi devstack@keystone.service[80234]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 23 11:40:35.346366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 11:40:35.346366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: from cryptography.utils import int_from_bytes Jun 23 11:40:35.347189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 11:40:35.347189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: from cryptography.utils import int_from_bytes Jun 23 11:40:35.348918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 11:40:35.348918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: from cryptography.utils import int_from_bytes Jun 23 11:40:35.349383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 11:40:35.349383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: from cryptography.utils import int_from_bytes Jun 23 11:40:35.395653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 11:40:35.395864 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 11:40:35.396025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] command line args: [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 11:40:35.396356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 11:40:35.396462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 11:40:35.396545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 11:40:35.396632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ================================================================================ {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 11:40:35.396757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] command line args: [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 11:40:35.396851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] admin_token = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.397033 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 11:40:35.397121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] config_dir = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.397219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ================================================================================ {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 11:40:35.397311 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.397410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] config_source = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.397489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] admin_token = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.397590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] debug = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.397722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] config_dir = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.397814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.398100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.398100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] config_source = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.398299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.398396 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] debug = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.398504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.398601 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.398737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.398842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.398946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.399105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.399220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.399329 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.399432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_config_append = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.399432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.399615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.399725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_dir = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.399828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.399930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_file = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.400033 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_config_append = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.400155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.400271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.400271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_dir = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.400455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.400579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_file = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.400662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.400779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.400895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.400990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.401065 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.401065 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.401245 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.401360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.401442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.401442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.401650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.401728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.401850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.401946 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.402037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.402111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.402236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.402318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.402390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.402505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.402597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.402679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.402757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.402879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.402962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.403092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.403311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.403453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.403592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.403671 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] publish_errors = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.403853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.404082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.404167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.404299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.404389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] publish_errors = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.404502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.404582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.404738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.404822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.404905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.404982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] standard_threads = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.405160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.405236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.405399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.405519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.405717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.406129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] standard_threads = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.406129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.406376 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] use_journal = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.407020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.407149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] use_json = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.407293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.407428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] use_stderr = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.407565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] use_journal = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.407700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] use_syslog = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.407833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] use_json = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.407833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] use_stderr = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.408081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.408168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] use_syslog = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.408168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 11:40:35.408309 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.408381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.408460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.408780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.408875 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.409254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.409341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.409531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.409732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.409819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.410048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.410207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.410286 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.410576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.410737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.410869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.411001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.411136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.411328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.411415 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.411588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.411690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.411848 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.411926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.412095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.412179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.412352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.412433 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.412600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.412699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.412887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.412969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.413144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.413231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.413400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.413485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.413652 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.413731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.413892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.413978 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.414159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.414276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.414397 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.414627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.414781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.414920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.415045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.415180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.415310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.415449 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.415596 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.415741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.415872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.416013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.416148 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.416301 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.416428 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.416549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.416788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.416921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.417050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.417219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.417300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.417424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.417501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.417619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.417695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.417695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.417886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.418007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.418084 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.418199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.418274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.418355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.418429 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.418499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.418616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.418729 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.418961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.419118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.419194 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.419413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.419499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.419645 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.419884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.419972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.420134 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.420357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.420436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.420592 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.420845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.420940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.421087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.421284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.421365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.421568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.421695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.421935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.422115 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.422785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.422917 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.423096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.423269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.423458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.423617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.423769 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.423920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.424072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.424268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.424410 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.424527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.424641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.424816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.424816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.425001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.425097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.425172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.425250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.425333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.425412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.425491 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.425491 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.425692 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.425776 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.425859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.425931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.426035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.426116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.426273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.426354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.426509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.426597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.426774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.426852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.427014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.427102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.427253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.427333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.427487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.427566 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.427725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.427813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.427962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.428039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.428215 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.428295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.428455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.428543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.428710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.428785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.428954 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.429024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.429195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.429268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.429435 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.429510 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.429672 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.429740 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.429934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.430021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.430148 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.430228 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.430397 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.430477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.430638 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.430719 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.430877 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.430959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.431116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.431197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.431356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.431438 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.431591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.431669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.431825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.431905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.432059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.432135 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.432285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.432365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.432515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.432595 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.432901 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.432976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.433067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.433186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.433267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.433429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.433510 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.433671 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.433759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.433917 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.434010 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.434150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.434271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.434452 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.434556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.434633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.434705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.434867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.434949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.435107 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.external = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.435197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.435339 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.435420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.435591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.435676 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.435832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.435910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.436072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.password = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.436151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.436302 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] auth.token = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.436382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.436542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.436636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.436838 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.436922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.external = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.437083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.437172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.437319 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.437399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.437568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.437650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.437848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.437934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.password = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.438080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] auth.token = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.438159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.438315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.438402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.438556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.438636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.438801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.438880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.439034 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.439114 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.439285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.439366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.439537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.439654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.439773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.439895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.440015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.440162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.440267 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.440421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.440509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.440663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.440808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.440941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.441022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.441665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.441784 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.441784 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.441936 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.442016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.442090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.442163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.442244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.442320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.442400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.442475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.442716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.442799 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.442969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.443077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.443241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.443459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.443539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.443720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.443809 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.443957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.444038 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.444195 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.444275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.444436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.444524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.444704 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.444810 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.444944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.445034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.445199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.445293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.445436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.445516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.445685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.445765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.445926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.446024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.446175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.446400 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.446481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.446731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.447015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.447210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.447299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.447363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.447423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.447483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.447564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.447640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.447738 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.447820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.447982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.448063 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.448227 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.448310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.448473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.448552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.448722 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.448804 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.448968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.449050 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.449216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.449296 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.449465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.449672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.449761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.450055 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.450149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.450224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.450308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.450993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.451107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.451186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.451260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.451334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.451407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.451480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.451566 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.451643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.451716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.451789 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.451862 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.451935 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.452017 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.452093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.452224 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.452306 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.452484 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.452563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.452752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.452901 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.452983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.453224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.453810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.454004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.454099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.454251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.454334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.454482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.454589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.454717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.454838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.454946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.455096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.455178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.455335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.455416 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.455590 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.455673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.455844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.455924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.456086 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.456165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.456334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.456414 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.456582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.456662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.456835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.456916 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.457068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.457151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.457304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.457384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.457545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.457625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.457781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.457863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.458020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.458100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.458256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.458337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.458488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.458570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.458727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.458805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.458976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.459052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.459204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.459276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.459436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.459515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.459685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.459774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.459932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.460012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.460168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.460247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.460398 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.460487 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.460635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.460776 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.460892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.460972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.461129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.461222 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.461360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.461441 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.461607 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.461695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.461854 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.461944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.462101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.462182 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.462344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.462424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.462583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.462671 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.462812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.462892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.463049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.463129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.463284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.463371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.463520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.463593 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.463751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.463820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.463984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.464062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.464236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.464316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.464483 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.464563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.464739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.464834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.464995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.465078 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.465227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.465307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.465473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.465562 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.465710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.465781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.465948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.466017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.466183 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.466270 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.466420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.466500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.466661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.466741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.466894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.466983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.467130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.467202 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.467371 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.467452 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.467606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.467693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.467855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.467935 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.468093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.468173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.468332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.468423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.468573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.468645 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.468832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.468938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.469647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.469737 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.469881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.469952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.470118 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.470199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.470356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] role.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.470445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.470605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] role.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.470686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.470836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.470917 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.471081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.471241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.471319 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.471399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.471563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.471645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.471800 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.471889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.472051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.472137 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.472289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.472369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.472531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.472620 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.472799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.472881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] role.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.473034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.473115 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] role.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.473280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.473363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.473524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.473605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.473773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.473857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.474020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.474103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.474268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.474460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.474540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.474698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.474787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.474948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.475030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.475197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.475277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.475444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.475533 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.475695 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.475776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.475929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.476010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.476181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.476270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.476430 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.476511 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.476687 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.476893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.476974 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.477135 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.477219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.477364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.477447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.477612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.477693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.477864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.477945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.478110 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.478188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.478346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.478425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.478584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.478665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.478815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.478895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.479052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.479140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.479290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.479370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.479522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.479601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.479757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.479847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.480010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.480091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.480256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.480335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.480495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.480592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.480758 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.480852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.481003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.481083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.481238 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.481326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.481478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.481559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.481717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.481797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.481963 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.482051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.482203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.482290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.482451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.482530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.482682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.482804 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.482929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.483062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.483228 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.483308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.483490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.connection = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.483583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.483736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.483827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.483985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.484065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.484227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.484307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.484471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.484556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.484760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.484867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.485001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.485079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.485241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.485335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.485478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.485561 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.485733 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.485826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.485961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.486163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.connection = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.486260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.486345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.486421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.486494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.486665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.486747 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.486901 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.486983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.487222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.487315 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.487390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.487463 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.487604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.487687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.487841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.487921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.488015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80244) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 11:40:35.488099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.488327 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.488576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.488842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.489099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.489340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.489587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.489924 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.490077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.490319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 11:40:35.490491 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80243) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 11:40:35.519187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.519408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.519632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.520966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.521202 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.521429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.528171 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.530118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.556897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.557120 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.559023 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.559253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.699508 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.700164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.748890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.749119 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.749633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.749874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.814471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.814699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.814788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.814924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.815012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.815143 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.815229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.815361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.815445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.815578 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.815664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.815794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.841530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.841765 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.841989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.842078 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.842206 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.842292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.842420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.842515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.842631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.842716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.842842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.842926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.843081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.843301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.854477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.854956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.867699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.867930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.868191 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.868409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.868492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.868623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.868736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.868857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.868944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.869071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80244) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.869156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.869302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.869521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.869737 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80243) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 11:40:35.886455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.886564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.886665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 11:40:35.886884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 11:40:35.886984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.887185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 11:40:35.887269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.887417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 11:40:35.887499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.887686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.887777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.887938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.888020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.888192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.888273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.888449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.899002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.899276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.899362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.899542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.899627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.899823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.899910 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.900119 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.908953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.909294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.912202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.912481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.912566 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.912783 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.912869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.913060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.913146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.913335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.913418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.913590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.913682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.913861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.913945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.914128 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.914210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.914393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.914485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.914660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.914743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.914921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.915003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.915187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.941843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.942471 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.943397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.943677 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.944108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.944390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.948941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.949328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.949616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.949756 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.950147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.950428 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.956891 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.957174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.957824 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.958105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.961646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.961931 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.962204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.962484 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.962637 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.962760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.962936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.963020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.963218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.963302 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.963501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.963586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.963790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.964071 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.964341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.964605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.982678 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.982950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.984229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.985056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.988070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.989835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.991693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.992087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.993517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.993914 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:35.997120 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:35.998928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.000822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.002565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.004457 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.004775 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.005059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.005342 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.005617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.005890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.006231 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.006530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.006816 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.007106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.007393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.007673 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.024849 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.025130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.025395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.025662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.026744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.027031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.027310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.027585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.034296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.035990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.036244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.036362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.037977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.038256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.039502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.039781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.040059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.040332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.040611 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.041481 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.041767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.042047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.042356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.042694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.054418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.054689 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.054983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.055267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.056574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.056881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.056977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.057168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.057253 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.057447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.057532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.059065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.059363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.059665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.075220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.077537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.078545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.080852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.081858 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.083587 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.084172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.085254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.085529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.085808 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.085926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.087571 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.087852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.088135 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.093134 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.095256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.096478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.098123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.098498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.098793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.099178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.099455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.101378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.101675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.102065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.102345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.108503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.108817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.109097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.111993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.112559 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.117529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80244) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.117807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.118085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.118359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.118623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.118986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.119266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.119533 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.119809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.120081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.120349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.120431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80243) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 11:40:36.120607 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.120728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.120892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.120976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.121163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80244) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.121296 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.121485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.121874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.122150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.122423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.122706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.122983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.123254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.123530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.123809 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.124075 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80243) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 11:40:36.159227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5586a956b9c0 pid: 80244 (default app) Jun 23 11:40:36.162420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5586a956b9c0 pid: 80243 (default app) Jun 23 11:40:36.162655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545429da-9088-4df8-8cea-91f637e7bf28 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:36.162947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545429da-9088-4df8-8cea-91f637e7bf28 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:36.163225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545429da-9088-4df8-8cea-91f637e7bf28 None None] PATH_INFO: `/v3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:36.167381 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1/1] 10.222.0.67 () {50 vars in 791 bytes} [Thu Jun 23 11:40:36 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:40:43.905144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dbcaf-64b1-4d67-9617-48f19861fc8f None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:43.905768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dbcaf-64b1-4d67-9617-48f19861fc8f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:43.905768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dbcaf-64b1-4d67-9617-48f19861fc8f None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:43.910375 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1/2] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:43.916497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab1a21-8a21-48b8-adad-49440b1ca0a2 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:43.917001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab1a21-8a21-48b8-adad-49440b1ca0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:43.917001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab1a21-8a21-48b8-adad-49440b1ca0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:43.986308 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: INFO dbcounter [None req-aaab1a21-8a21-48b8-adad-49440b1ca0a2 None None] Registered counter for database keystone Jun 23 11:40:44.008645 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG oslo_db.sqlalchemy.engines [None req-aaab1a21-8a21-48b8-adad-49440b1ca0a2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80244) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 11:40:44.015103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writer thread running {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 11:40:44.199501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG passlib.handlers.bcrypt [None req-aaab1a21-8a21-48b8-adad-49440b1ca0a2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80244) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 11:40:44.199849 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG passlib.handlers.bcrypt [None req-aaab1a21-8a21-48b8-adad-49440b1ca0a2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80244) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 11:40:44.255460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-aaab1a21-8a21-48b8-adad-49440b1ca0a2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:44.257137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-aaab1a21-8a21-48b8-adad-49440b1ca0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:44.386999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 2/3] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 23 11:40:44.527481 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dbcaf-64b1-4d67-9617-48f19861fc8f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:44.598451 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: INFO dbcounter [None req-ab6dbcaf-64b1-4d67-9617-48f19861fc8f None None] Registered counter for database keystone Jun 23 11:40:44.617539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG oslo_db.sqlalchemy.engines [None req-ab6dbcaf-64b1-4d67-9617-48f19861fc8f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80243) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 11:40:44.624836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writer thread running {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 11:40:44.745771 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dbcaf-64b1-4d67-9617-48f19861fc8f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:44.750194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d8952-66d3-46fc-9cee-4729e05bc979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:44.750862 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d8952-66d3-46fc-9cee-4729e05bc979 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:44.751151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d8952-66d3-46fc-9cee-4729e05bc979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:44.751430 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d8952-66d3-46fc-9cee-4729e05bc979 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:44.759710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d8952-66d3-46fc-9cee-4729e05bc979 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:44.812144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: WARNING keystone.common.rbac_enforcer.enforcer [None req-827d8952-66d3-46fc-9cee-4729e05bc979 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 11:40:44.812427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d8952-66d3-46fc-9cee-4729e05bc979 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:44.822688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 2/4] 10.222.0.67 () {58 vars in 1324 bytes} [Thu Jun 23 11:40:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:40:44.829246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab1a21-8a21-48b8-adad-49440b1ca0a2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:44.843522 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab1a21-8a21-48b8-adad-49440b1ca0a2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:44.848009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2b750-a7c7-4ffa-b287-ab3704ae0d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:44.849141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2b750-a7c7-4ffa-b287-ab3704ae0d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:44.849438 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2b750-a7c7-4ffa-b287-ab3704ae0d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:44.849721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2b750-a7c7-4ffa-b287-ab3704ae0d18 admin admin] PATH_INFO: `/v3/projects/1e503e3101124c66a9726948b602dc87` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:44.853880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2b750-a7c7-4ffa-b287-ab3704ae0d18 admin admin] RBAC: Authorizing `identity:get_project(project_id=1e503e3101124c66a9726948b602dc87)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:44.904794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b5d2b750-a7c7-4ffa-b287-ab3704ae0d18 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 11:40:44.905138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2b750-a7c7-4ffa-b287-ab3704ae0d18 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:44.910033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 3/5] 10.222.0.67 () {56 vars in 1207 bytes} [Thu Jun 23 11:40:44 2022] GET /identity/v3/projects/1e503e3101124c66a9726948b602dc87 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:40:46.901678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047bf6d-8726-4dfc-991d-f351d5f43da0 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:46.902215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047bf6d-8726-4dfc-991d-f351d5f43da0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:46.902215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047bf6d-8726-4dfc-991d-f351d5f43da0 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:46.906640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 3/6] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:46.912121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a34097-8cf8-4275-a13f-f6395f5a667d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:46.912498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a34097-8cf8-4275-a13f-f6395f5a667d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:46.912692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a34097-8cf8-4275-a13f-f6395f5a667d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:46.931726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-84a34097-8cf8-4275-a13f-f6395f5a667d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:46.933215 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-84a34097-8cf8-4275-a13f-f6395f5a667d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:46.941564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 4/7] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 23 11:40:46.964192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7047bf6d-8726-4dfc-991d-f351d5f43da0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:46.976240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7047bf6d-8726-4dfc-991d-f351d5f43da0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:46.980456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d257e3c-2967-4427-9afa-f385a023e8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:46.981114 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d257e3c-2967-4427-9afa-f385a023e8ef admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:46.981405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d257e3c-2967-4427-9afa-f385a023e8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:46.981688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d257e3c-2967-4427-9afa-f385a023e8ef admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:46.989907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d257e3c-2967-4427-9afa-f385a023e8ef admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:46.991674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d257e3c-2967-4427-9afa-f385a023e8ef admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:47.001282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 4/8] 10.222.0.67 () {58 vars in 1324 bytes} [Thu Jun 23 11:40:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:40:47.006785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a34097-8cf8-4275-a13f-f6395f5a667d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:47.016027 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a34097-8cf8-4275-a13f-f6395f5a667d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:47.020473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbd797-32eb-4935-8c1d-159686adb456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:47.021366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbd797-32eb-4935-8c1d-159686adb456 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:47.021656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbd797-32eb-4935-8c1d-159686adb456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:47.021938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbd797-32eb-4935-8c1d-159686adb456 admin admin] PATH_INFO: `/v3/users/86b831b5471b48e6bf2bf9ce771279b2` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:47.023343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbd797-32eb-4935-8c1d-159686adb456 admin admin] RBAC: Authorizing `identity:get_user(user_id=86b831b5471b48e6bf2bf9ce771279b2)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:47.025212 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbd797-32eb-4935-8c1d-159686adb456 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:47.026791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 5/9] 10.222.0.67 () {56 vars in 1198 bytes} [Thu Jun 23 11:40:47 2022] GET /identity/v3/users/86b831b5471b48e6bf2bf9ce771279b2 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:40:49.086546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b29210-307f-48a4-b3f6-ecdd7207b6a7 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:49.087079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b29210-307f-48a4-b3f6-ecdd7207b6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:49.087164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b29210-307f-48a4-b3f6-ecdd7207b6a7 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:49.088474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 5/10] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:49.093737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5838f7c-a629-4d84-a1cc-bdc391021e7f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:49.094677 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5838f7c-a629-4d84-a1cc-bdc391021e7f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:49.094817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5838f7c-a629-4d84-a1cc-bdc391021e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:49.111234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f5838f7c-a629-4d84-a1cc-bdc391021e7f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:49.112432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f5838f7c-a629-4d84-a1cc-bdc391021e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:49.133378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db059c88-4b00-4549-bdc3-0e7814ccc885 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:49.133672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db059c88-4b00-4549-bdc3-0e7814ccc885 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:49.133961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db059c88-4b00-4549-bdc3-0e7814ccc885 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:49.135327 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 6/11] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:40:49.140353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2cbc6-bc7b-47eb-8034-5e22739ac8f3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:49.141222 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2cbc6-bc7b-47eb-8034-5e22739ac8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:49.141479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2cbc6-bc7b-47eb-8034-5e22739ac8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:49.181221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG passlib.handlers.bcrypt [None req-96c2cbc6-bc7b-47eb-8034-5e22739ac8f3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80243) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 11:40:49.181646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG passlib.handlers.bcrypt [None req-96c2cbc6-bc7b-47eb-8034-5e22739ac8f3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80243) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 11:40:49.194301 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 6/12] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:40:49.217575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5838f7c-a629-4d84-a1cc-bdc391021e7f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:49.227933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-96c2cbc6-bc7b-47eb-8034-5e22739ac8f3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:49.229648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-96c2cbc6-bc7b-47eb-8034-5e22739ac8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:49.241300 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:49.325502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5838f7c-a629-4d84-a1cc-bdc391021e7f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:49.332466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 7/13] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:40:49.354662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2cbc6-bc7b-47eb-8034-5e22739ac8f3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:49.360384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e708fa-1a2d-4195-bb97-a973b6973554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:49.361225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e708fa-1a2d-4195-bb97-a973b6973554 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:49.361517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e708fa-1a2d-4195-bb97-a973b6973554 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:49.361802 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e708fa-1a2d-4195-bb97-a973b6973554 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:49.373835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e708fa-1a2d-4195-bb97-a973b6973554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-24e708fa-1a2d-4195-bb97-a973b6973554 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:40:49.385370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:40:49.386497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:40:49.387440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 7/14] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:40:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:40:49.392369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e708fa-1a2d-4195-bb97-a973b6973554 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:49.461036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2cbc6-bc7b-47eb-8034-5e22739ac8f3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:49.463369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:49.476116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:49.495969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e708fa-1a2d-4195-bb97-a973b6973554 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:49.498124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751992c5-47d0-4c6f-a075-096be089dee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:49.499015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751992c5-47d0-4c6f-a075-096be089dee8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:49.499303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751992c5-47d0-4c6f-a075-096be089dee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:49.499580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751992c5-47d0-4c6f-a075-096be089dee8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:49.529599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98204d-401e-4f2f-b63b-d397c90bda6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:49.530322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98204d-401e-4f2f-b63b-d397c90bda6e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:49.530614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98204d-401e-4f2f-b63b-d397c90bda6e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:49.530897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98204d-401e-4f2f-b63b-d397c90bda6e None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:49.532463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98204d-401e-4f2f-b63b-d397c90bda6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:40:49.532809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98204d-401e-4f2f-b63b-d397c90bda6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:49.534209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98204d-401e-4f2f-b63b-d397c90bda6e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:49.545806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 8/15] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:40:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:40:49.551752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98204d-401e-4f2f-b63b-d397c90bda6e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:49.567971 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751992c5-47d0-4c6f-a075-096be089dee8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:49.569548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751992c5-47d0-4c6f-a075-096be089dee8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:49.649180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98204d-401e-4f2f-b63b-d397c90bda6e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:49.667026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 8/16] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:40:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:40:49.673168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751992c5-47d0-4c6f-a075-096be089dee8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:49.675950 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:49.686245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e854f-b2b5-494d-bc5d-efd5c9640e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:49.687121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e854f-b2b5-494d-bc5d-efd5c9640e70 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:49.687417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e854f-b2b5-494d-bc5d-efd5c9640e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:49.687705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e854f-b2b5-494d-bc5d-efd5c9640e70 None admin] PATH_INFO: `/v3/users/86b831b5471b48e6bf2bf9ce771279b2` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:49.688969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e854f-b2b5-494d-bc5d-efd5c9640e70 None admin] RBAC: Authorizing `identity:get_user(user_id=86b831b5471b48e6bf2bf9ce771279b2)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:49.690439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e854f-b2b5-494d-bc5d-efd5c9640e70 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:49.692078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 9/17] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:40:49 2022] GET /identity/v3/users/86b831b5471b48e6bf2bf9ce771279b2 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:40:49.697298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e854f-b2b5-494d-bc5d-efd5c9640e70 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:49.711822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:49.772168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751992c5-47d0-4c6f-a075-096be089dee8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:49.799352 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e854f-b2b5-494d-bc5d-efd5c9640e70 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:49.807436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84489783-93fe-4e83-8500-7e6f20f64d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:49.808230 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84489783-93fe-4e83-8500-7e6f20f64d0b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:49.808573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84489783-93fe-4e83-8500-7e6f20f64d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:49.808827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84489783-93fe-4e83-8500-7e6f20f64d0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:49.828019 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84489783-93fe-4e83-8500-7e6f20f64d0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:49.829549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84489783-93fe-4e83-8500-7e6f20f64d0b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:49.841564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd292d0-885f-464a-838b-ffebad48da9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:49.842407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd292d0-885f-464a-838b-ffebad48da9e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:49.842694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd292d0-885f-464a-838b-ffebad48da9e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:49.842969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd292d0-885f-464a-838b-ffebad48da9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:49.844172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd292d0-885f-464a-838b-ffebad48da9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:49.845647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd292d0-885f-464a-838b-ffebad48da9e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:49.847248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 10/18] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:40:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-84489783-93fe-4e83-8500-7e6f20f64d0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:40:49.850674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:40:49.851499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:40:49.852308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:40:49.852308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:40:49.852308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:40:49.852308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:40:49.852308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 9/19] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:40:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:40:49.853696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd292d0-885f-464a-838b-ffebad48da9e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:49.857313 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84489783-93fe-4e83-8500-7e6f20f64d0b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:49.914461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:49.935688 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:49.956691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd292d0-885f-464a-838b-ffebad48da9e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:49.959450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84489783-93fe-4e83-8500-7e6f20f64d0b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:49.991282 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7ec77-d52c-4c4c-971c-bba7bf91812c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:49.991940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7ec77-d52c-4c4c-971c-bba7bf91812c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:49.992225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7ec77-d52c-4c4c-971c-bba7bf91812c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:49.992504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7ec77-d52c-4c4c-971c-bba7bf91812c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:49.993404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7ec77-d52c-4c4c-971c-bba7bf91812c None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.domain.id=default, user.id=86b831b5471b48e6bf2bf9ce771279b2) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:40:49.993728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7ec77-d52c-4c4c-971c-bba7bf91812c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:49.995167 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7ec77-d52c-4c4c-971c-bba7bf91812c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:49.995280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e0c52-027d-427c-b6db-b006850615da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:49.995894 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e0c52-027d-427c-b6db-b006850615da None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:49.996188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e0c52-027d-427c-b6db-b006850615da None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:49.996473 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e0c52-027d-427c-b6db-b006850615da None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:49.997253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e0c52-027d-427c-b6db-b006850615da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:40:49.997566 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e0c52-027d-427c-b6db-b006850615da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:49.998953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e0c52-027d-427c-b6db-b006850615da None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:50.002868 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 11/20] 10.222.0.67 () {56 vars in 1322 bytes} [Thu Jun 23 11:40:49 2022] GET /identity/v3/role_assignments?user.id=86b831b5471b48e6bf2bf9ce771279b2&scope.domain.id=default&role.id=569707378794437b93d168fc8bb9cbc3 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:40:50.018984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 10/21] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:40:52.304483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf942420-7ede-4913-b4ca-4f60e7a63484 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:52.305526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf942420-7ede-4913-b4ca-4f60e7a63484 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:52.306035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf942420-7ede-4913-b4ca-4f60e7a63484 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:52.307637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 12/22] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:52.325628 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836728e-6a83-4c06-899d-ef7b2f39a00d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:52.326141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836728e-6a83-4c06-899d-ef7b2f39a00d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:52.326217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836728e-6a83-4c06-899d-ef7b2f39a00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:52.351558 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b836728e-6a83-4c06-899d-ef7b2f39a00d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:52.352939 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b836728e-6a83-4c06-899d-ef7b2f39a00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:52.430240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 11/23] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:40:52.465696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf942420-7ede-4913-b4ca-4f60e7a63484 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:52.487258 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a72b51-efbd-4eda-b62b-44278dbfcfef None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:52.487865 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a72b51-efbd-4eda-b62b-44278dbfcfef None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:52.488367 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a72b51-efbd-4eda-b62b-44278dbfcfef None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:52.489967 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 12/24] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:40:52.495739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efbe69-b26b-48cb-a83a-ae7834128899 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:52.496316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efbe69-b26b-48cb-a83a-ae7834128899 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:52.496814 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19efbe69-b26b-48cb-a83a-ae7834128899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:52.533710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-19efbe69-b26b-48cb-a83a-ae7834128899 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:52.535281 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-19efbe69-b26b-48cb-a83a-ae7834128899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:52.639091 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:52.660402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf942420-7ede-4913-b4ca-4f60e7a63484 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:52.694801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 13/25] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:40:52.733455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a1ede-217a-4d68-a6c4-93740e6992e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:52.734843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a1ede-217a-4d68-a6c4-93740e6992e5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:52.735146 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a1ede-217a-4d68-a6c4-93740e6992e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:52.735430 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a1ede-217a-4d68-a6c4-93740e6992e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:52.736443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efbe69-b26b-48cb-a83a-ae7834128899 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:52.762231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a1ede-217a-4d68-a6c4-93740e6992e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-9c5a1ede-217a-4d68-a6c4-93740e6992e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:40:52.770081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:40:52.770966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:40:52.771718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 13/26] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:40:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:40:52.778955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a1ede-217a-4d68-a6c4-93740e6992e5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:52.799303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:52.901525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19efbe69-b26b-48cb-a83a-ae7834128899 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:52.961771 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5891a92e-75a5-4ad8-9f3d-4a80cbca3a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:52.962573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891a92e-75a5-4ad8-9f3d-4a80cbca3a38 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:52.962861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891a92e-75a5-4ad8-9f3d-4a80cbca3a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:52.963137 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891a92e-75a5-4ad8-9f3d-4a80cbca3a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:52.974481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a1ede-217a-4d68-a6c4-93740e6992e5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:52.988691 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891a92e-75a5-4ad8-9f3d-4a80cbca3a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:52.990199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5891a92e-75a5-4ad8-9f3d-4a80cbca3a38 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-5891a92e-75a5-4ad8-9f3d-4a80cbca3a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:40:53.019956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:40:53.020890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:40:53.021706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:40:53.021706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:40:53.021706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:40:53.021706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:40:53.023872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 14/27] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:40:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:40:53.025423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419acb0-701c-4e08-b2df-e95ee2d26bf3 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:53.026023 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419acb0-701c-4e08-b2df-e95ee2d26bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:53.026461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419acb0-701c-4e08-b2df-e95ee2d26bf3 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:53.028066 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 15/28] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:53.030079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419acb0-701c-4e08-b2df-e95ee2d26bf3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:53.048649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cf3f9-5e29-43b5-9812-adcdce7dc9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:53.049700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cf3f9-5e29-43b5-9812-adcdce7dc9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:53.050196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cf3f9-5e29-43b5-9812-adcdce7dc9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:53.050671 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cf3f9-5e29-43b5-9812-adcdce7dc9c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:53.051625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cf3f9-5e29-43b5-9812-adcdce7dc9c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:40:53.052133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cf3f9-5e29-43b5-9812-adcdce7dc9c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:53.053774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cf3f9-5e29-43b5-9812-adcdce7dc9c1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:53.067310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 14/29] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:40:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:40:53.069624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae6a80-c0ca-43b8-8a00-b55d1f3b3972 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:53.071538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae6a80-c0ca-43b8-8a00-b55d1f3b3972 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:53.072794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae6a80-c0ca-43b8-8a00-b55d1f3b3972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:53.097028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:53.104781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6aae6a80-c0ca-43b8-8a00-b55d1f3b3972 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:53.105956 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6aae6a80-c0ca-43b8-8a00-b55d1f3b3972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:53.181916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:53.196841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 15/30] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:40:53.197728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae6a80-c0ca-43b8-8a00-b55d1f3b3972 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:53.213234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419acb0-701c-4e08-b2df-e95ee2d26bf3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:53.265702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f9ac63-5378-45d7-91d1-47970d750600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:53.267169 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9ac63-5378-45d7-91d1-47970d750600 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:53.267708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9ac63-5378-45d7-91d1-47970d750600 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:53.268136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f9ac63-5378-45d7-91d1-47970d750600 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:53.269298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9ac63-5378-45d7-91d1-47970d750600 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:40:53.270126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9ac63-5378-45d7-91d1-47970d750600 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:53.271650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f9ac63-5378-45d7-91d1-47970d750600 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:53.288957 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 16/31] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:40:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:40:53.290899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95373b18-d86c-401d-81d3-e2e8d839fe47 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:53.291609 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95373b18-d86c-401d-81d3-e2e8d839fe47 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:53.292043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95373b18-d86c-401d-81d3-e2e8d839fe47 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:53.294051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 17/32] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:53.298600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95373b18-d86c-401d-81d3-e2e8d839fe47 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:53.338107 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae6a80-c0ca-43b8-8a00-b55d1f3b3972 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:53.401106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ba7da-378f-48e7-a560-1341e3e2e292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:53.403279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ba7da-378f-48e7-a560-1341e3e2e292 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:53.403789 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ba7da-378f-48e7-a560-1341e3e2e292 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:53.404507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ba7da-378f-48e7-a560-1341e3e2e292 None admin] PATH_INFO: `/v3/users/86b831b5471b48e6bf2bf9ce771279b2` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:53.407550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ba7da-378f-48e7-a560-1341e3e2e292 None admin] RBAC: Authorizing `identity:get_user(user_id=86b831b5471b48e6bf2bf9ce771279b2)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:53.409343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ba7da-378f-48e7-a560-1341e3e2e292 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:53.412043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 16/33] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:40:53 2022] GET /identity/v3/users/86b831b5471b48e6bf2bf9ce771279b2 => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:40:53.413189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ba7da-378f-48e7-a560-1341e3e2e292 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:53.416404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:53.434659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95373b18-d86c-401d-81d3-e2e8d839fe47 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:53.479319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8bf9c0-9f8a-4dbe-a838-4cd5a072c66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:53.480429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bf9c0-9f8a-4dbe-a838-4cd5a072c66c None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:53.480974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bf9c0-9f8a-4dbe-a838-4cd5a072c66c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:53.481730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8bf9c0-9f8a-4dbe-a838-4cd5a072c66c None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:53.483528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bf9c0-9f8a-4dbe-a838-4cd5a072c66c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:53.486174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8bf9c0-9f8a-4dbe-a838-4cd5a072c66c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:53.512444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:53.565808 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ba7da-378f-48e7-a560-1341e3e2e292 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:53.613015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab0ec4-ac3d-42dc-b17f-a6fc1f1ab52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:53.613911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab0ec4-ac3d-42dc-b17f-a6fc1f1ab52e None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:53.614367 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab0ec4-ac3d-42dc-b17f-a6fc1f1ab52e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:53.614844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfab0ec4-ac3d-42dc-b17f-a6fc1f1ab52e None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:53.615698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab0ec4-ac3d-42dc-b17f-a6fc1f1ab52e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:53.617367 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfab0ec4-ac3d-42dc-b17f-a6fc1f1ab52e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:53.643953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 18/34] 10.222.0.67 () {60 vars in 1131 bytes} [Thu Jun 23 11:40:53 2022] POST /identity/v3/roles => generated 225 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:40:53.646095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293cb00-3770-4506-84d7-4fd1d93ef129 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:53.646631 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293cb00-3770-4506-84d7-4fd1d93ef129 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:53.647202 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c293cb00-3770-4506-84d7-4fd1d93ef129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:53.699634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c293cb00-3770-4506-84d7-4fd1d93ef129 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:53.701632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c293cb00-3770-4506-84d7-4fd1d93ef129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:53.837468 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 19/35] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:40:53.838784 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293cb00-3770-4506-84d7-4fd1d93ef129 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:53.896748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 17/36] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:40:53 2022] POST /identity/v3/projects => generated 301 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:40:53.898166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab0ec4-ac3d-42dc-b17f-a6fc1f1ab52e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:53.969216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:53.999839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c293cb00-3770-4506-84d7-4fd1d93ef129 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:54.040927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527decf-ebf6-4a98-bf50-d8b61f85fa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:54.041754 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527decf-ebf6-4a98-bf50-d8b61f85fa55 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:54.042033 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527decf-ebf6-4a98-bf50-d8b61f85fa55 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:54.042497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527decf-ebf6-4a98-bf50-d8b61f85fa55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:54.044125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527decf-ebf6-4a98-bf50-d8b61f85fa55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:54.045807 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527decf-ebf6-4a98-bf50-d8b61f85fa55 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:54.048146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 20/37] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:40:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:40:54.049464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010a651-d339-4e28-9eaa-f229256d047e None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:54.049980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010a651-d339-4e28-9eaa-f229256d047e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:54.050429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010a651-d339-4e28-9eaa-f229256d047e None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:54.052109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 21/38] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:54.057116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfab0ec4-ac3d-42dc-b17f-a6fc1f1ab52e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:54.057301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d06b6-132a-4405-87c5-23a259ba7291 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:54.057438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d06b6-132a-4405-87c5-23a259ba7291 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:54.058778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d06b6-132a-4405-87c5-23a259ba7291 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:54.060041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 22/39] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:54.063505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010641b-d647-444b-8a27-10310a1d0f04 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:54.063999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010641b-d647-444b-8a27-10310a1d0f04 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:54.064560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010641b-d647-444b-8a27-10310a1d0f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:54.088349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:54.102928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9010641b-d647-444b-8a27-10310a1d0f04 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:54.104159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9010641b-d647-444b-8a27-10310a1d0f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:54.126740 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc577a6-d84e-4a95-a91c-d1d9f375107f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:54.127095 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc577a6-d84e-4a95-a91c-d1d9f375107f None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:54.127390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc577a6-d84e-4a95-a91c-d1d9f375107f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:54.127678 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc577a6-d84e-4a95-a91c-d1d9f375107f None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:54.128457 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc577a6-d84e-4a95-a91c-d1d9f375107f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:54.130048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc577a6-d84e-4a95-a91c-d1d9f375107f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:54.146179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 18/40] 10.222.0.67 () {60 vars in 1131 bytes} [Thu Jun 23 11:40:53 2022] POST /identity/v3/roles => generated 219 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:40:54.147251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc577a6-d84e-4a95-a91c-d1d9f375107f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:54.201741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 23/41] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:40:54.204118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd1c82-d460-4373-b077-d8cf4edb08bf None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:54.204570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd1c82-d460-4373-b077-d8cf4edb08bf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:54.205064 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd1c82-d460-4373-b077-d8cf4edb08bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:54.226953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-38cd1c82-d460-4373-b077-d8cf4edb08bf None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:54.228187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-38cd1c82-d460-4373-b077-d8cf4edb08bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:54.284439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc577a6-d84e-4a95-a91c-d1d9f375107f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:54.304377 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 24/42] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:40:54.305372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd1c82-d460-4373-b077-d8cf4edb08bf None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:54.315832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:54.323458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712267c-8f11-4677-b22d-56359dde5ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:54.324336 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712267c-8f11-4677-b22d-56359dde5ddd None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:54.324629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712267c-8f11-4677-b22d-56359dde5ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:54.324974 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712267c-8f11-4677-b22d-56359dde5ddd None admin] PATH_INFO: `/v3/domains/default/users/86b831b5471b48e6bf2bf9ce771279b2/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:54.327214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712267c-8f11-4677-b22d-56359dde5ddd None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=86b831b5471b48e6bf2bf9ce771279b2, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:54.329502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712267c-8f11-4677-b22d-56359dde5ddd None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:54.348699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 19/43] 10.222.0.67 () {58 vars in 1361 bytes} [Thu Jun 23 11:40:54 2022] PUT /identity/v3/domains/default/users/86b831b5471b48e6bf2bf9ce771279b2/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:40:54.349553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712267c-8f11-4677-b22d-56359dde5ddd None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:54.398695 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:54.442046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd1c82-d460-4373-b077-d8cf4edb08bf None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:54.481984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb78488-8f23-4acf-b02f-7ddc7e118dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:54.482722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb78488-8f23-4acf-b02f-7ddc7e118dcf None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:54.483018 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb78488-8f23-4acf-b02f-7ddc7e118dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:54.483314 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb78488-8f23-4acf-b02f-7ddc7e118dcf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:54.484592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb78488-8f23-4acf-b02f-7ddc7e118dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:54.485274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712267c-8f11-4677-b22d-56359dde5ddd None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:54.486172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb78488-8f23-4acf-b02f-7ddc7e118dcf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:54.487895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 25/44] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:40:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:40:54.493846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb78488-8f23-4acf-b02f-7ddc7e118dcf None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:54.635169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e0109-e042-4d74-beb7-246bbe456b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:54.636169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e0109-e042-4d74-beb7-246bbe456b17 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:54.636610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e0109-e042-4d74-beb7-246bbe456b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:54.637083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e0109-e042-4d74-beb7-246bbe456b17 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:54.638018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e0109-e042-4d74-beb7-246bbe456b17 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:54.639612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e0109-e042-4d74-beb7-246bbe456b17 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:54.654552 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 20/45] 10.222.0.67 () {60 vars in 1131 bytes} [Thu Jun 23 11:40:54 2022] POST /identity/v3/roles => generated 223 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:40:54.693082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:54.724254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb78488-8f23-4acf-b02f-7ddc7e118dcf None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:54.763608 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973ac9e-5efe-4722-824f-5572ba0f89c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:54.764242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973ac9e-5efe-4722-824f-5572ba0f89c8 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:54.764537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973ac9e-5efe-4722-824f-5572ba0f89c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:54.764852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973ac9e-5efe-4722-824f-5572ba0f89c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:54.765627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973ac9e-5efe-4722-824f-5572ba0f89c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:54.767134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973ac9e-5efe-4722-824f-5572ba0f89c8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:54.804702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 26/46] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:40:54 2022] POST /identity/v3/projects => generated 312 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:40:56.428961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9aa88d-57a7-4c6e-9ddb-35194f87aef0 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:56.429561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9aa88d-57a7-4c6e-9ddb-35194f87aef0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:56.429561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9aa88d-57a7-4c6e-9ddb-35194f87aef0 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:56.430913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 21/47] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:56.436220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc551e7-8ea7-493e-af44-1162d39b14eb None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:56.436628 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc551e7-8ea7-493e-af44-1162d39b14eb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:56.436834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc551e7-8ea7-493e-af44-1162d39b14eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:56.454828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2fc551e7-8ea7-493e-af44-1162d39b14eb None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:56.456057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2fc551e7-8ea7-493e-af44-1162d39b14eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:56.523356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 27/48] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:40:56.547966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9aa88d-57a7-4c6e-9ddb-35194f87aef0 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:56.642346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:56.649505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9aa88d-57a7-4c6e-9ddb-35194f87aef0 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:56.685597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa61258-90d2-4f2c-83ca-87e2ae48fc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:56.686329 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa61258-90d2-4f2c-83ca-87e2ae48fc20 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:56.686623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa61258-90d2-4f2c-83ca-87e2ae48fc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:56.686903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa61258-90d2-4f2c-83ca-87e2ae48fc20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:56.699191 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa61258-90d2-4f2c-83ca-87e2ae48fc20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-7aa61258-90d2-4f2c-83ca-87e2ae48fc20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:40:56.707015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:40:56.707914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:40:56.708797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 22/49] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:40:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:40:56.714218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc551e7-8ea7-493e-af44-1162d39b14eb None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:56.772122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:56.816962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc551e7-8ea7-493e-af44-1162d39b14eb None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:56.821394 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e2d0-af13-440c-ad27-e8c0367bd77b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:56.821686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e2d0-af13-440c-ad27-e8c0367bd77b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:56.821971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e2d0-af13-440c-ad27-e8c0367bd77b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:56.823349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 23/50] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:56.828130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e51ce-d37b-48a3-b0b0-285d2d7adbec None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:56.828424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e51ce-d37b-48a3-b0b0-285d2d7adbec None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:56.828734 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e51ce-d37b-48a3-b0b0-285d2d7adbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:56.846547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2f2e51ce-d37b-48a3-b0b0-285d2d7adbec None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:56.848092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2f2e51ce-d37b-48a3-b0b0-285d2d7adbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:56.852136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c3d42-5a4b-4db8-8335-0f3719d3faf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:56.852851 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c3d42-5a4b-4db8-8335-0f3719d3faf0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:56.853208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c3d42-5a4b-4db8-8335-0f3719d3faf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:56.853498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c3d42-5a4b-4db8-8335-0f3719d3faf0 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:56.854280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c3d42-5a4b-4db8-8335-0f3719d3faf0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:40:56.854609 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c3d42-5a4b-4db8-8335-0f3719d3faf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:56.856059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c3d42-5a4b-4db8-8335-0f3719d3faf0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:56.868641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 28/51] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:40:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:40:56.874087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c3d42-5a4b-4db8-8335-0f3719d3faf0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:56.919953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 24/52] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:40:56.943762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e51ce-d37b-48a3-b0b0-285d2d7adbec None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:56.974767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c3d42-5a4b-4db8-8335-0f3719d3faf0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:56.992609 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:57.010418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca6978-8afb-4cc2-b623-fea4369cd7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:57.011335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca6978-8afb-4cc2-b623-fea4369cd7dd None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:57.011636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca6978-8afb-4cc2-b623-fea4369cd7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:57.011919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fca6978-8afb-4cc2-b623-fea4369cd7dd None admin] PATH_INFO: `/v3/users/86b831b5471b48e6bf2bf9ce771279b2` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:57.013409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca6978-8afb-4cc2-b623-fea4369cd7dd None admin] RBAC: Authorizing `identity:get_user(user_id=86b831b5471b48e6bf2bf9ce771279b2)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:57.014884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fca6978-8afb-4cc2-b623-fea4369cd7dd None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:57.017136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 29/53] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:40:56 2022] GET /identity/v3/users/86b831b5471b48e6bf2bf9ce771279b2 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:40:57.022446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca6978-8afb-4cc2-b623-fea4369cd7dd None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:57.026973 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:57.043317 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e51ce-d37b-48a3-b0b0-285d2d7adbec None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:57.090279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0452fa5-3a26-4f08-a4e2-a84720665f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:57.091070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0452fa5-3a26-4f08-a4e2-a84720665f7f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:57.091357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0452fa5-3a26-4f08-a4e2-a84720665f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:57.091640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0452fa5-3a26-4f08-a4e2-a84720665f7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:57.092855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0452fa5-3a26-4f08-a4e2-a84720665f7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:57.094324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0452fa5-3a26-4f08-a4e2-a84720665f7f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:57.095950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 25/54] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:40:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:40:57.101390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0452fa5-3a26-4f08-a4e2-a84720665f7f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:57.132885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fca6978-8afb-4cc2-b623-fea4369cd7dd None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:57.166112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df3fe0-1846-4488-b77e-cf783719636f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:57.166904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df3fe0-1846-4488-b77e-cf783719636f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:57.167196 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df3fe0-1846-4488-b77e-cf783719636f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:57.167480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df3fe0-1846-4488-b77e-cf783719636f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:57.168685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df3fe0-1846-4488-b77e-cf783719636f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:57.170148 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df3fe0-1846-4488-b77e-cf783719636f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:57.171784 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 30/55] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:40:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:40:57.177371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df3fe0-1846-4488-b77e-cf783719636f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:57.199952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0452fa5-3a26-4f08-a4e2-a84720665f7f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:57.232742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:57.234747 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7271a8-0462-47db-8d49-b3e3c8c700e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:57.235439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7271a8-0462-47db-8d49-b3e3c8c700e9 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:57.235738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7271a8-0462-47db-8d49-b3e3c8c700e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:57.236031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7271a8-0462-47db-8d49-b3e3c8c700e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:57.236793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7271a8-0462-47db-8d49-b3e3c8c700e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:57.238292 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7271a8-0462-47db-8d49-b3e3c8c700e9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:57.264629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 26/56] 10.222.0.67 () {60 vars in 1140 bytes} [Thu Jun 23 11:40:57 2022] POST /identity/v3/projects => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:40:57.281701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df3fe0-1846-4488-b77e-cf783719636f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:57.319716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199168f2-1dac-41a8-8e0a-2e58538c19bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:57.320408 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199168f2-1dac-41a8-8e0a-2e58538c19bc None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:57.320732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199168f2-1dac-41a8-8e0a-2e58538c19bc None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:57.321021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199168f2-1dac-41a8-8e0a-2e58538c19bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:57.321894 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199168f2-1dac-41a8-8e0a-2e58538c19bc None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.domain.id=default, user.id=86b831b5471b48e6bf2bf9ce771279b2) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:40:57.322217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199168f2-1dac-41a8-8e0a-2e58538c19bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:57.323710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199168f2-1dac-41a8-8e0a-2e58538c19bc None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:57.334148 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 31/57] 10.222.0.67 () {56 vars in 1322 bytes} [Thu Jun 23 11:40:57 2022] GET /identity/v3/role_assignments?user.id=86b831b5471b48e6bf2bf9ce771279b2&scope.domain.id=default&role.id=569707378794437b93d168fc8bb9cbc3 => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:40:59.315466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e0474-d710-4c2b-8e8b-6a3d8bf4201a None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:59.315977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e0474-d710-4c2b-8e8b-6a3d8bf4201a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:59.316063 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e0474-d710-4c2b-8e8b-6a3d8bf4201a None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:59.317496 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 27/58] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:40:59.322565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad873afa-200e-439e-ac5b-83714b88b09f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:59.322974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad873afa-200e-439e-ac5b-83714b88b09f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:59.323139 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad873afa-200e-439e-ac5b-83714b88b09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:59.341392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ad873afa-200e-439e-ac5b-83714b88b09f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:40:59.342621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ad873afa-200e-439e-ac5b-83714b88b09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:40:59.406689 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 32/59] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:40:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:40:59.434731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e0474-d710-4c2b-8e8b-6a3d8bf4201a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:59.445033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:59.539875 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e0474-d710-4c2b-8e8b-6a3d8bf4201a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:59.577245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e229c-083e-4e06-af3c-8c753fe9afa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:59.578475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e229c-083e-4e06-af3c-8c753fe9afa4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:59.578764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e229c-083e-4e06-af3c-8c753fe9afa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:59.579052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e229c-083e-4e06-af3c-8c753fe9afa4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:59.580254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e229c-083e-4e06-af3c-8c753fe9afa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:59.581772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e229c-083e-4e06-af3c-8c753fe9afa4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:59.583422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 28/60] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:40:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:40:59.589191 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad873afa-200e-439e-ac5b-83714b88b09f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:40:59.640093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:40:59.693309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad873afa-200e-439e-ac5b-83714b88b09f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:40:59.727600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c593379-3f88-439e-a50f-8fa1d71853fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:40:59.728272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c593379-3f88-439e-a50f-8fa1d71853fc None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:40:59.728568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c593379-3f88-439e-a50f-8fa1d71853fc None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:40:59.728879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c593379-3f88-439e-a50f-8fa1d71853fc None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:40:59.729602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c593379-3f88-439e-a50f-8fa1d71853fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:40:59.731090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c593379-3f88-439e-a50f-8fa1d71853fc None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:40:59.760606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 33/61] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:40:59 2022] POST /identity/v3/users => generated 275 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:41:02.835146 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159e52a-28b1-4728-8380-f64a538a6940 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:02.835671 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159e52a-28b1-4728-8380-f64a538a6940 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:02.835749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159e52a-28b1-4728-8380-f64a538a6940 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:02.837289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 29/62] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:02.845506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50cf8e-7a12-4442-928d-c12e73645433 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:02.846380 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50cf8e-7a12-4442-928d-c12e73645433 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:02.846899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50cf8e-7a12-4442-928d-c12e73645433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:02.871372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9c50cf8e-7a12-4442-928d-c12e73645433 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:02.872933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9c50cf8e-7a12-4442-928d-c12e73645433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:02.972190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 34/63] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:03.007270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159e52a-28b1-4728-8380-f64a538a6940 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:03.015372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6c8bb-d07e-4eb2-9d02-ac87113aed26 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.015967 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6c8bb-d07e-4eb2-9d02-ac87113aed26 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.016452 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad6c8bb-d07e-4eb2-9d02-ac87113aed26 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.018088 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 35/64] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:41:03.031955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e4355-5a0a-451a-93c9-c16b0b37a0d8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.032575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e4355-5a0a-451a-93c9-c16b0b37a0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.033082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e4355-5a0a-451a-93c9-c16b0b37a0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.051227 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-015e4355-5a0a-451a-93c9-c16b0b37a0d8 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:03.052831 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-015e4355-5a0a-451a-93c9-c16b0b37a0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:03.097684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:03.120646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:03.151039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159e52a-28b1-4728-8380-f64a538a6940 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:03.188460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 36/65] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:03.192592 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74a869-2f07-45b0-9a84-cd0895c6e9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:03.193595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74a869-2f07-45b0-9a84-cd0895c6e9aa None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.193889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74a869-2f07-45b0-9a84-cd0895c6e9aa None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.194167 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74a869-2f07-45b0-9a84-cd0895c6e9aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.210608 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e4355-5a0a-451a-93c9-c16b0b37a0d8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:03.217074 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74a869-2f07-45b0-9a84-cd0895c6e9aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-5b74a869-2f07-45b0-9a84-cd0895c6e9aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:03.225190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:03.226066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:03.227340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 30/66] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:03.239510 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74a869-2f07-45b0-9a84-cd0895c6e9aa None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:03.378462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e4355-5a0a-451a-93c9-c16b0b37a0d8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:03.388589 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74a869-2f07-45b0-9a84-cd0895c6e9aa None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:03.424101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62134cc9-86fb-45d5-982f-2417944889ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:03.425092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62134cc9-86fb-45d5-982f-2417944889ec None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.425488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62134cc9-86fb-45d5-982f-2417944889ec None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.425888 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62134cc9-86fb-45d5-982f-2417944889ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.427646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957faf1-c0e6-433d-af9e-3032d2b82bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:03.428576 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957faf1-c0e6-433d-af9e-3032d2b82bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.429025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957faf1-c0e6-433d-af9e-3032d2b82bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.429470 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957faf1-c0e6-433d-af9e-3032d2b82bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.430472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957faf1-c0e6-433d-af9e-3032d2b82bd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:03.430924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957faf1-c0e6-433d-af9e-3032d2b82bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:03.432470 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957faf1-c0e6-433d-af9e-3032d2b82bd6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:03.434335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:03.441682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62134cc9-86fb-45d5-982f-2417944889ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:03.444408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 31/67] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:03.445913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75320d06-fef5-4da9-80ae-3e5cbd833ac6 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.446387 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75320d06-fef5-4da9-80ae-3e5cbd833ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.446847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75320d06-fef5-4da9-80ae-3e5cbd833ac6 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.448261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 32/68] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-62134cc9-86fb-45d5-982f-2417944889ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:03.453629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:03.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:03.455316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:03.455316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:03.455316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:03.455316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:03.455316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:03.455316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:03.455316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:03.455316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:03.455316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:03.455316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:03.455316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:03.455316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:03.455899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75320d06-fef5-4da9-80ae-3e5cbd833ac6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:03.455966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 37/69] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:41:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:41:03.457664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb97305-bf7a-448f-97b5-a32d00c9a83d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.458936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb97305-bf7a-448f-97b5-a32d00c9a83d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.459643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb97305-bf7a-448f-97b5-a32d00c9a83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.477820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-efb97305-bf7a-448f-97b5-a32d00c9a83d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:03.479303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-efb97305-bf7a-448f-97b5-a32d00c9a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:03.502036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:03.572751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 38/70] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:03.573941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb97305-bf7a-448f-97b5-a32d00c9a83d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:03.589390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75320d06-fef5-4da9-80ae-3e5cbd833ac6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:03.636008 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a01a78-e260-44da-a61b-ba28a0dfee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:03.637226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a01a78-e260-44da-a61b-ba28a0dfee58 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.637689 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a01a78-e260-44da-a61b-ba28a0dfee58 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.638165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a01a78-e260-44da-a61b-ba28a0dfee58 None admin] PATH_INFO: `/v3/users/6dcf44525c2549deba53d9ec8dbe03ad` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.653353 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a01a78-e260-44da-a61b-ba28a0dfee58 None admin] RBAC: Authorizing `identity:get_user(user_id=6dcf44525c2549deba53d9ec8dbe03ad)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:03.654817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a01a78-e260-44da-a61b-ba28a0dfee58 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:03.656978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 33/71] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:03 2022] GET /identity/v3/users/6dcf44525c2549deba53d9ec8dbe03ad => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:03.657696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a01a78-e260-44da-a61b-ba28a0dfee58 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:03.697349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb97305-bf7a-448f-97b5-a32d00c9a83d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:03.712383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:03.731497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be1f6c-6ebb-4003-b9da-695c781ec280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:03.732096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be1f6c-6ebb-4003-b9da-695c781ec280 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.732398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be1f6c-6ebb-4003-b9da-695c781ec280 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.732693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be1f6c-6ebb-4003-b9da-695c781ec280 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.733477 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be1f6c-6ebb-4003-b9da-695c781ec280 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:03.733795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be1f6c-6ebb-4003-b9da-695c781ec280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:03.735248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be1f6c-6ebb-4003-b9da-695c781ec280 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:03.743431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 39/72] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:03.744890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a99d9-a7af-44e5-b6c2-51e8d353035a None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.745342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a99d9-a7af-44e5-b6c2-51e8d353035a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.745626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a99d9-a7af-44e5-b6c2-51e8d353035a None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.747029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 40/73] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:03.748253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75d908-ea8e-40f2-9279-a2b03924d52a None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.748734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75d908-ea8e-40f2-9279-a2b03924d52a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.749162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75d908-ea8e-40f2-9279-a2b03924d52a None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.751069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 41/74] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:03.752791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75d908-ea8e-40f2-9279-a2b03924d52a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:03.761360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a01a78-e260-44da-a61b-ba28a0dfee58 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:03.771494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:03.795910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4d2cb-5ed1-42e1-9e33-d9357c8b8cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:03.796770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4d2cb-5ed1-42e1-9e33-d9357c8b8cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.797057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4d2cb-5ed1-42e1-9e33-d9357c8b8cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.797337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4d2cb-5ed1-42e1-9e33-d9357c8b8cd8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.807363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4d2cb-5ed1-42e1-9e33-d9357c8b8cd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-faa4d2cb-5ed1-42e1-9e33-d9357c8b8cd8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:03.816117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:03.817014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:03.817989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 34/75] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:03.818814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95722a58-97ae-4901-8d5f-a2ad07077bb6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.818814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95722a58-97ae-4901-8d5f-a2ad07077bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.819074 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95722a58-97ae-4901-8d5f-a2ad07077bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.835614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-95722a58-97ae-4901-8d5f-a2ad07077bb6 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:03.836857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-95722a58-97ae-4901-8d5f-a2ad07077bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:03.857332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75d908-ea8e-40f2-9279-a2b03924d52a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:03.894108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0edcdc-b6ee-4d2c-b46a-ef1d5db7adfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:03.894768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0edcdc-b6ee-4d2c-b46a-ef1d5db7adfe None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:03.895065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0edcdc-b6ee-4d2c-b46a-ef1d5db7adfe None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:03.895353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0edcdc-b6ee-4d2c-b46a-ef1d5db7adfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:03.902752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 35/76] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:03.903610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95722a58-97ae-4901-8d5f-a2ad07077bb6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:03.966811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:03.967617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0edcdc-b6ee-4d2c-b46a-ef1d5db7adfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:03.969449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0edcdc-b6ee-4d2c-b46a-ef1d5db7adfe None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:03.987365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:04.022846 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95722a58-97ae-4901-8d5f-a2ad07077bb6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:04.057724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72635a-8bb9-41b6-9da6-8dc0d8f887a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:04.058594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72635a-8bb9-41b6-9da6-8dc0d8f887a9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:04.058889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72635a-8bb9-41b6-9da6-8dc0d8f887a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:04.059175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72635a-8bb9-41b6-9da6-8dc0d8f887a9 None admin] PATH_INFO: `/v3/users/86b831b5471b48e6bf2bf9ce771279b2` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:04.060390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72635a-8bb9-41b6-9da6-8dc0d8f887a9 None admin] RBAC: Authorizing `identity:get_user(user_id=86b831b5471b48e6bf2bf9ce771279b2)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:04.061888 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da72635a-8bb9-41b6-9da6-8dc0d8f887a9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:04.064082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 36/77] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:03 2022] GET /identity/v3/users/86b831b5471b48e6bf2bf9ce771279b2 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:04.065254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9abb5-ca1d-4b27-964f-7ff3294de0a7 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:04.065548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9abb5-ca1d-4b27-964f-7ff3294de0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:04.065833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9abb5-ca1d-4b27-964f-7ff3294de0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:04.075242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 42/78] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:04.077651 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0edcdc-b6ee-4d2c-b46a-ef1d5db7adfe None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:04.082120 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-fcc9abb5-ca1d-4b27-964f-7ff3294de0a7 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:04.083330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fcc9abb5-ca1d-4b27-964f-7ff3294de0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:04.148447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 37/79] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:04.158072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9abb5-ca1d-4b27-964f-7ff3294de0a7 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:04.176921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0edcdc-b6ee-4d2c-b46a-ef1d5db7adfe None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:04.193534 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:04.199236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:04.211685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d8b7d-e3e8-4192-8d35-907ab5cf47b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:04.212356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d8b7d-e3e8-4192-8d35-907ab5cf47b8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:04.212640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d8b7d-e3e8-4192-8d35-907ab5cf47b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:04.212953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d8b7d-e3e8-4192-8d35-907ab5cf47b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:04.213701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d8b7d-e3e8-4192-8d35-907ab5cf47b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:04.214011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d8b7d-e3e8-4192-8d35-907ab5cf47b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:04.215404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d8b7d-e3e8-4192-8d35-907ab5cf47b8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:04.223491 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 43/80] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:04.224429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d8b7d-e3e8-4192-8d35-907ab5cf47b8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:04.258499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9abb5-ca1d-4b27-964f-7ff3294de0a7 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:04.291586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e301c14-57e1-429f-8b63-5d2da4e526a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:04.292382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301c14-57e1-429f-8b63-5d2da4e526a6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:04.292691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301c14-57e1-429f-8b63-5d2da4e526a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:04.292984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e301c14-57e1-429f-8b63-5d2da4e526a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:04.303702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e301c14-57e1-429f-8b63-5d2da4e526a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-6e301c14-57e1-429f-8b63-5d2da4e526a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:04.311440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:04.313043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:04.314069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 38/81] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:04.314945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e301c14-57e1-429f-8b63-5d2da4e526a6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:04.326569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d8b7d-e3e8-4192-8d35-907ab5cf47b8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:04.362028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e0bf6-1532-4162-a8c9-c25ac332066b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:04.362683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e0bf6-1532-4162-a8c9-c25ac332066b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:04.362977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e0bf6-1532-4162-a8c9-c25ac332066b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:04.363294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e0bf6-1532-4162-a8c9-c25ac332066b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:04.418465 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:04.425333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e301c14-57e1-429f-8b63-5d2da4e526a6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:04.426907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:04.446097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e0bf6-1532-4162-a8c9-c25ac332066b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:04.447752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496e0bf6-1532-4162-a8c9-c25ac332066b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:04.471679 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc677c-1376-41be-aa5a-64dafd70493b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:04.472559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc677c-1376-41be-aa5a-64dafd70493b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:04.472889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc677c-1376-41be-aa5a-64dafd70493b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:04.473695 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dc677c-1376-41be-aa5a-64dafd70493b None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:04.474959 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc677c-1376-41be-aa5a-64dafd70493b None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:04.476702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dc677c-1376-41be-aa5a-64dafd70493b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:04.478202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 39/82] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:04 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:04.478979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc677c-1376-41be-aa5a-64dafd70493b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:04.555767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 44/83] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:04.556592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e0bf6-1532-4162-a8c9-c25ac332066b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:04.588399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dc677c-1376-41be-aa5a-64dafd70493b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:04.625602 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb95ed9-9f6d-4916-85b4-31071d1c4d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:04.626402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb95ed9-9f6d-4916-85b4-31071d1c4d2c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:04.626695 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb95ed9-9f6d-4916-85b4-31071d1c4d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:04.626979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb95ed9-9f6d-4916-85b4-31071d1c4d2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:04.628175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb95ed9-9f6d-4916-85b4-31071d1c4d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:04.629677 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb95ed9-9f6d-4916-85b4-31071d1c4d2c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:04.631475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 40/84] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:04.632079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb95ed9-9f6d-4916-85b4-31071d1c4d2c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:04.648023 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:04.676532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e0bf6-1532-4162-a8c9-c25ac332066b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:04.681037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:04.710627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d1916-65b8-49c1-90bf-0f401e366830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:04.711504 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d1916-65b8-49c1-90bf-0f401e366830 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:04.711800 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d1916-65b8-49c1-90bf-0f401e366830 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:04.712085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d1916-65b8-49c1-90bf-0f401e366830 None admin] PATH_INFO: `/v3/users/6dcf44525c2549deba53d9ec8dbe03ad` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:04.713319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d1916-65b8-49c1-90bf-0f401e366830 None admin] RBAC: Authorizing `identity:get_user(user_id=6dcf44525c2549deba53d9ec8dbe03ad)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:04.714798 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d1916-65b8-49c1-90bf-0f401e366830 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:04.716460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 45/85] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:04 2022] GET /identity/v3/users/6dcf44525c2549deba53d9ec8dbe03ad => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:04.717260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d1916-65b8-49c1-90bf-0f401e366830 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:04.741331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb95ed9-9f6d-4916-85b4-31071d1c4d2c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:04.775725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524d516-7aa9-4cf1-8a74-6029a2437203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:04.776400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524d516-7aa9-4cf1-8a74-6029a2437203 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:04.776717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524d516-7aa9-4cf1-8a74-6029a2437203 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:04.777001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9524d516-7aa9-4cf1-8a74-6029a2437203 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:04.777766 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524d516-7aa9-4cf1-8a74-6029a2437203 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:04.778083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524d516-7aa9-4cf1-8a74-6029a2437203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:04.779482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9524d516-7aa9-4cf1-8a74-6029a2437203 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:04.789817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 41/86] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:04.790628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524d516-7aa9-4cf1-8a74-6029a2437203 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:04.820166 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d1916-65b8-49c1-90bf-0f401e366830 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:04.853422 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546831b7-4078-44e3-8738-a701eb0e0152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:04.854078 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546831b7-4078-44e3-8738-a701eb0e0152 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:04.854368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546831b7-4078-44e3-8738-a701eb0e0152 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:04.854649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546831b7-4078-44e3-8738-a701eb0e0152 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:04.855514 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546831b7-4078-44e3-8738-a701eb0e0152 None admin] RBAC: Adding query filter params (role.id=20703235b4c543bc9c51fb7c26da270d, scope.project.id=ea37fde41a0c43a790da11fb742b5490, user.id=6dcf44525c2549deba53d9ec8dbe03ad) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:04.855834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546831b7-4078-44e3-8738-a701eb0e0152 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:04.857346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546831b7-4078-44e3-8738-a701eb0e0152 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:04.862758 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 46/87] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:04 2022] GET /identity/v3/role_assignments?user.id=6dcf44525c2549deba53d9ec8dbe03ad&scope.project.id=ea37fde41a0c43a790da11fb742b5490&role.id=20703235b4c543bc9c51fb7c26da270d => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:04.865552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546831b7-4078-44e3-8738-a701eb0e0152 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:04.880616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:04.897761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:04.901876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9524d516-7aa9-4cf1-8a74-6029a2437203 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:04.944160 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd039fa5-993e-4fc3-b1ea-10bd99bcb669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:04.945087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd039fa5-993e-4fc3-b1ea-10bd99bcb669 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:04.945382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd039fa5-993e-4fc3-b1ea-10bd99bcb669 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:04.945874 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd039fa5-993e-4fc3-b1ea-10bd99bcb669 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:04.947836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd039fa5-993e-4fc3-b1ea-10bd99bcb669 None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:04.949708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd039fa5-993e-4fc3-b1ea-10bd99bcb669 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:04.958028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 42/88] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:04 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:04.958028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd039fa5-993e-4fc3-b1ea-10bd99bcb669 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:04.989070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546831b7-4078-44e3-8738-a701eb0e0152 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:05.031853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec8d07-c0e5-45f6-a070-b00d2191973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:05.032538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec8d07-c0e5-45f6-a070-b00d2191973b None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:05.032923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec8d07-c0e5-45f6-a070-b00d2191973b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:05.033205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec8d07-c0e5-45f6-a070-b00d2191973b None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:05.033936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec8d07-c0e5-45f6-a070-b00d2191973b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:05.035396 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec8d07-c0e5-45f6-a070-b00d2191973b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:05.057008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 47/89] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:41:04 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:41:05.062102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec8d07-c0e5-45f6-a070-b00d2191973b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:05.075409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd039fa5-993e-4fc3-b1ea-10bd99bcb669 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:05.122262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8038375-1986-435b-946e-4c10caf5ab4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:05.123030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8038375-1986-435b-946e-4c10caf5ab4e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:05.123390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8038375-1986-435b-946e-4c10caf5ab4e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:05.123763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8038375-1986-435b-946e-4c10caf5ab4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:05.158260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:05.165357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:05.204561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8038375-1986-435b-946e-4c10caf5ab4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:05.206229 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8038375-1986-435b-946e-4c10caf5ab4e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:05.208435 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec8d07-c0e5-45f6-a070-b00d2191973b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:05.258349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076a485-ed71-4e49-a6ce-0d7d73101508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:05.259253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076a485-ed71-4e49-a6ce-0d7d73101508 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:05.259543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076a485-ed71-4e49-a6ce-0d7d73101508 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:05.259825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076a485-ed71-4e49-a6ce-0d7d73101508 None admin] PATH_INFO: `/v3/users/6dcf44525c2549deba53d9ec8dbe03ad` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:05.261672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076a485-ed71-4e49-a6ce-0d7d73101508 None admin] RBAC: Authorizing `identity:get_user(user_id=6dcf44525c2549deba53d9ec8dbe03ad)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:05.264110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076a485-ed71-4e49-a6ce-0d7d73101508 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:05.266413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 48/90] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:05 2022] GET /identity/v3/users/6dcf44525c2549deba53d9ec8dbe03ad => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:05.267197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076a485-ed71-4e49-a6ce-0d7d73101508 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:05.350807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 43/91] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:05.351677 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8038375-1986-435b-946e-4c10caf5ab4e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:05.401806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076a485-ed71-4e49-a6ce-0d7d73101508 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:05.433687 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:05.445441 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dce70-1050-41f9-a0f3-22b24a8682d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:05.446344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dce70-1050-41f9-a0f3-22b24a8682d4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:05.446472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dce70-1050-41f9-a0f3-22b24a8682d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:05.446684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951dce70-1050-41f9-a0f3-22b24a8682d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:05.447631 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dce70-1050-41f9-a0f3-22b24a8682d4 None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=ea37fde41a0c43a790da11fb742b5490, user.id=86b831b5471b48e6bf2bf9ce771279b2) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:05.447951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dce70-1050-41f9-a0f3-22b24a8682d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:05.449449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951dce70-1050-41f9-a0f3-22b24a8682d4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:05.451622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:05.459722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 49/92] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:05 2022] GET /identity/v3/role_assignments?user.id=86b831b5471b48e6bf2bf9ce771279b2&scope.project.id=ea37fde41a0c43a790da11fb742b5490&role.id=569707378794437b93d168fc8bb9cbc3 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:05.461152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dce70-1050-41f9-a0f3-22b24a8682d4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:05.482016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8038375-1986-435b-946e-4c10caf5ab4e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:05.526832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410c5f7-0c8e-4122-bf6f-4e9d3354461d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:05.527604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410c5f7-0c8e-4122-bf6f-4e9d3354461d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:05.527964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410c5f7-0c8e-4122-bf6f-4e9d3354461d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:05.528759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410c5f7-0c8e-4122-bf6f-4e9d3354461d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:05.595759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951dce70-1050-41f9-a0f3-22b24a8682d4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:05.607563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410c5f7-0c8e-4122-bf6f-4e9d3354461d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:05.609736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410c5f7-0c8e-4122-bf6f-4e9d3354461d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:05.635439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308f3f6-9b4f-4fee-a582-84636b8232f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:05.636836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308f3f6-9b4f-4fee-a582-84636b8232f0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:05.637398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308f3f6-9b4f-4fee-a582-84636b8232f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:05.637857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308f3f6-9b4f-4fee-a582-84636b8232f0 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:05.639320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308f3f6-9b4f-4fee-a582-84636b8232f0 None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:05.641252 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a308f3f6-9b4f-4fee-a582-84636b8232f0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:05.643335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 50/93] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:05 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:05.649630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308f3f6-9b4f-4fee-a582-84636b8232f0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:05.683569 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:05.725171 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 44/94] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:05.730628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410c5f7-0c8e-4122-bf6f-4e9d3354461d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:05.752780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:05.817912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a308f3f6-9b4f-4fee-a582-84636b8232f0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:05.857665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746ba6e-7f50-48a7-ac1b-ddfa25d38d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:05.858247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746ba6e-7f50-48a7-ac1b-ddfa25d38d0d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:05.858535 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746ba6e-7f50-48a7-ac1b-ddfa25d38d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:05.858813 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746ba6e-7f50-48a7-ac1b-ddfa25d38d0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:05.859836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746ba6e-7f50-48a7-ac1b-ddfa25d38d0d None admin] RBAC: Adding query filter params (role.id=2d0e0231af5d4e51b80de6bb65e8b02b, scope.project.id=ea37fde41a0c43a790da11fb742b5490, user.id=6dcf44525c2549deba53d9ec8dbe03ad) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:05.860157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746ba6e-7f50-48a7-ac1b-ddfa25d38d0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:05.861522 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410c5f7-0c8e-4122-bf6f-4e9d3354461d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:05.863021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746ba6e-7f50-48a7-ac1b-ddfa25d38d0d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:05.869684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 51/95] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:05 2022] GET /identity/v3/role_assignments?user.id=6dcf44525c2549deba53d9ec8dbe03ad&scope.project.id=ea37fde41a0c43a790da11fb742b5490&role.id=2d0e0231af5d4e51b80de6bb65e8b02b => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:05.917400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ce21a-a29a-417c-8793-7a153ca3861c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:05.918309 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ce21a-a29a-417c-8793-7a153ca3861c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:05.918601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ce21a-a29a-417c-8793-7a153ca3861c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:05.918882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ce21a-a29a-417c-8793-7a153ca3861c None admin] PATH_INFO: `/v3/projects/bad446a9e7c7496382a00b02db480740` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:05.920285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ce21a-a29a-417c-8793-7a153ca3861c None admin] RBAC: Authorizing `identity:get_project(project_id=bad446a9e7c7496382a00b02db480740)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:05.922361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ce21a-a29a-417c-8793-7a153ca3861c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:05.924123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 45/96] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:05 2022] GET /identity/v3/projects/bad446a9e7c7496382a00b02db480740 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:05.930668 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746ba6e-7f50-48a7-ac1b-ddfa25d38d0d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:06.085033 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746ba6e-7f50-48a7-ac1b-ddfa25d38d0d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:06.121167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:06.127788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2700c6c-3698-4d69-a6bd-46ce86d9cbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:06.128805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2700c6c-3698-4d69-a6bd-46ce86d9cbac None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:06.129320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2700c6c-3698-4d69-a6bd-46ce86d9cbac None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:06.129803 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2700c6c-3698-4d69-a6bd-46ce86d9cbac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:06.130840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2700c6c-3698-4d69-a6bd-46ce86d9cbac None admin] RBAC: Adding query filter params (role.id=20703235b4c543bc9c51fb7c26da270d, scope.project.id=bad446a9e7c7496382a00b02db480740, user.id=6dcf44525c2549deba53d9ec8dbe03ad) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:06.131373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2700c6c-3698-4d69-a6bd-46ce86d9cbac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:06.133103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2700c6c-3698-4d69-a6bd-46ce86d9cbac None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:06.139663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 52/97] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:05 2022] GET /identity/v3/role_assignments?user.id=6dcf44525c2549deba53d9ec8dbe03ad&scope.project.id=bad446a9e7c7496382a00b02db480740&role.id=20703235b4c543bc9c51fb7c26da270d => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:07.949061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903b9a3-2389-468a-a851-e784dd3b99eb None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:07.950054 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903b9a3-2389-468a-a851-e784dd3b99eb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:07.950570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903b9a3-2389-468a-a851-e784dd3b99eb None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:07.952169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 46/98] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:07.958455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3442a82-fe7b-4bd7-9465-ed15f484c9a1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:07.959299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3442a82-fe7b-4bd7-9465-ed15f484c9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:07.959785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3442a82-fe7b-4bd7-9465-ed15f484c9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:07.981729 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d3442a82-fe7b-4bd7-9465-ed15f484c9a1 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:07.983236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d3442a82-fe7b-4bd7-9465-ed15f484c9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:08.074320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 53/99] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:08.112068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903b9a3-2389-468a-a851-e784dd3b99eb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:08.165413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:08.249539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903b9a3-2389-468a-a851-e784dd3b99eb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:08.259402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4943d-5e9e-4fc7-80e6-f442227fe9ac None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:08.259685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4943d-5e9e-4fc7-80e6-f442227fe9ac None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:08.259976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4943d-5e9e-4fc7-80e6-f442227fe9ac None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:08.261420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 54/100] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:41:08.287619 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ae4e6-05c0-42cb-8670-0f26f52764aa None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:08.287913 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ae4e6-05c0-42cb-8670-0f26f52764aa None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:08.288200 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ae4e6-05c0-42cb-8670-0f26f52764aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:08.312818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215ed1a-0cda-4c4f-bb90-92dde6abe138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:08.313762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215ed1a-0cda-4c4f-bb90-92dde6abe138 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:08.314002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215ed1a-0cda-4c4f-bb90-92dde6abe138 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:08.314292 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215ed1a-0cda-4c4f-bb90-92dde6abe138 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:08.317432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215ed1a-0cda-4c4f-bb90-92dde6abe138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:08.318922 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215ed1a-0cda-4c4f-bb90-92dde6abe138 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:08.319150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b09ae4e6-05c0-42cb-8670-0f26f52764aa None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:08.320354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b09ae4e6-05c0-42cb-8670-0f26f52764aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:08.320611 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 47/101] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:08.322002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725b414-4d54-452e-968d-dd863d7cbbd2 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:08.322331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725b414-4d54-452e-968d-dd863d7cbbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:08.322706 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725b414-4d54-452e-968d-dd863d7cbbd2 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:08.324116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 48/102] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:08.329405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725b414-4d54-452e-968d-dd863d7cbbd2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:08.427501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 55/103] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:08.428963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6029fd-4d7b-49e2-b993-0939aedfd771 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:08.429338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6029fd-4d7b-49e2-b993-0939aedfd771 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:08.429699 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6029fd-4d7b-49e2-b993-0939aedfd771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:08.447773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4d6029fd-4d7b-49e2-b993-0939aedfd771 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:08.449006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4d6029fd-4d7b-49e2-b993-0939aedfd771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:08.532784 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725b414-4d54-452e-968d-dd863d7cbbd2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:08.537551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:08.542614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:08.572937 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b4ac9-5a30-40bc-9309-6532fa4b2148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:08.573606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b4ac9-5a30-40bc-9309-6532fa4b2148 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:08.573919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b4ac9-5a30-40bc-9309-6532fa4b2148 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:08.574189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93b4ac9-5a30-40bc-9309-6532fa4b2148 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:08.574887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b4ac9-5a30-40bc-9309-6532fa4b2148 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:08.575393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 56/104] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:08.576188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6029fd-4d7b-49e2-b993-0939aedfd771 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:08.576358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93b4ac9-5a30-40bc-9309-6532fa4b2148 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:08.623403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 49/105] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:41:08 2022] POST /identity/v3/projects => generated 302 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:41:08.624261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b4ac9-5a30-40bc-9309-6532fa4b2148 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:08.735703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93b4ac9-5a30-40bc-9309-6532fa4b2148 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:08.767324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6029fd-4d7b-49e2-b993-0939aedfd771 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:08.778324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1fd5d-f514-4150-b952-e6427c1ca1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:08.779147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1fd5d-f514-4150-b952-e6427c1ca1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:08.779428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1fd5d-f514-4150-b952-e6427c1ca1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:08.779712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1fd5d-f514-4150-b952-e6427c1ca1f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:08.800297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1fd5d-f514-4150-b952-e6427c1ca1f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-75e1fd5d-f514-4150-b952-e6427c1ca1f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:08.808224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:08.809161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:08.809926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:08.809926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:08.809926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:08.809926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:08.809926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:08.809926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:08.809926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:08.809926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:08.809926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:08.809926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:08.809926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:08.809926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:08.816511 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 50/106] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:08.826088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1fd5d-f514-4150-b952-e6427c1ca1f0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:08.845417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8040dac-828b-4cc5-b101-1ee0f04786c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:08.846271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8040dac-828b-4cc5-b101-1ee0f04786c1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:08.846534 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8040dac-828b-4cc5-b101-1ee0f04786c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:08.846817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8040dac-828b-4cc5-b101-1ee0f04786c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:08.867127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:08.875499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8040dac-828b-4cc5-b101-1ee0f04786c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-b8040dac-828b-4cc5-b101-1ee0f04786c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:08.883546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:08.885646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:08.887508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:08.887508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:08.887508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:08.887508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:08.887508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:08.887508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:08.887508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:08.887508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:08.887508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:08.887508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:08.887508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:08.887508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:08.895391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 57/107] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:08.901106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8040dac-828b-4cc5-b101-1ee0f04786c1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:08.976960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:09.007104 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1fd5d-f514-4150-b952-e6427c1ca1f0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:09.040406 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8040dac-828b-4cc5-b101-1ee0f04786c1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:09.046347 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5b0d7e-35d3-49f2-bb5c-6108a362ba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:09.047155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b0d7e-35d3-49f2-bb5c-6108a362ba96 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.047447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b0d7e-35d3-49f2-bb5c-6108a362ba96 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.047750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b0d7e-35d3-49f2-bb5c-6108a362ba96 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.048506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b0d7e-35d3-49f2-bb5c-6108a362ba96 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:09.048856 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b0d7e-35d3-49f2-bb5c-6108a362ba96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:09.050475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5b0d7e-35d3-49f2-bb5c-6108a362ba96 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:09.059767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 51/108] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:09.061412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef2803-5ba9-43f4-a618-083b2241b336 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.061645 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef2803-5ba9-43f4-a618-083b2241b336 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.061928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef2803-5ba9-43f4-a618-083b2241b336 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.064022 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 52/109] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:09.065310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fa5fc-4c98-45cb-9242-d6407bd52ec3 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.065700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fa5fc-4c98-45cb-9242-d6407bd52ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.065998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fa5fc-4c98-45cb-9242-d6407bd52ec3 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.067256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 53/110] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:09.072651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1b0e8-65a4-40fb-b608-29e74f69777b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.073242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1b0e8-65a4-40fb-b608-29e74f69777b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.073412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed1b0e8-65a4-40fb-b608-29e74f69777b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.080783 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b95e02-79da-4e22-8107-56ef7ec905ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:09.081472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b95e02-79da-4e22-8107-56ef7ec905ea None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.081768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b95e02-79da-4e22-8107-56ef7ec905ea None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.082053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b95e02-79da-4e22-8107-56ef7ec905ea None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.082819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b95e02-79da-4e22-8107-56ef7ec905ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:09.083133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b95e02-79da-4e22-8107-56ef7ec905ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:09.084635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b95e02-79da-4e22-8107-56ef7ec905ea None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:09.091214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-aed1b0e8-65a4-40fb-b608-29e74f69777b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:09.092448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-aed1b0e8-65a4-40fb-b608-29e74f69777b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:09.094527 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 58/111] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:09.096607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54890f-17c4-46ab-bb6a-b9d5251030f4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.096929 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54890f-17c4-46ab-bb6a-b9d5251030f4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.097218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54890f-17c4-46ab-bb6a-b9d5251030f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.118490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5f54890f-17c4-46ab-bb6a-b9d5251030f4 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:09.120215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5f54890f-17c4-46ab-bb6a-b9d5251030f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:09.166770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 54/112] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:09.173643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1b0e8-65a4-40fb-b608-29e74f69777b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:09.189792 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:09.204694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 59/113] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:09.204694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54890f-17c4-46ab-bb6a-b9d5251030f4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:09.244287 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:09.303343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed1b0e8-65a4-40fb-b608-29e74f69777b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:09.314652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54890f-17c4-46ab-bb6a-b9d5251030f4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:09.339266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a739d-458e-42d6-b0b2-dff66c3e4529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:09.340145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a739d-458e-42d6-b0b2-dff66c3e4529 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.340433 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a739d-458e-42d6-b0b2-dff66c3e4529 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.340744 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a739d-458e-42d6-b0b2-dff66c3e4529 None admin] PATH_INFO: `/v3/users/6dcf44525c2549deba53d9ec8dbe03ad` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.342001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a739d-458e-42d6-b0b2-dff66c3e4529 None admin] RBAC: Authorizing `identity:get_user(user_id=6dcf44525c2549deba53d9ec8dbe03ad)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:09.343506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a739d-458e-42d6-b0b2-dff66c3e4529 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:09.345203 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 55/114] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:09 2022] GET /identity/v3/users/6dcf44525c2549deba53d9ec8dbe03ad => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:09.349762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0129ceb-e4b5-45a9-8fce-9f837598c220 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.349762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0129ceb-e4b5-45a9-8fce-9f837598c220 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.349762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0129ceb-e4b5-45a9-8fce-9f837598c220 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.349762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 56/115] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:09.349762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0129ceb-e4b5-45a9-8fce-9f837598c220 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:09.357801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecc5f1-0ff0-4063-9250-ba25d8e8d328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:09.358305 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecc5f1-0ff0-4063-9250-ba25d8e8d328 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.358597 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecc5f1-0ff0-4063-9250-ba25d8e8d328 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.358883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecc5f1-0ff0-4063-9250-ba25d8e8d328 None admin] PATH_INFO: `/v3/users/5408af77976f48d5a5cad9ff9a0d20ff` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.376788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecc5f1-0ff0-4063-9250-ba25d8e8d328 None admin] RBAC: Authorizing `identity:get_user(user_id=5408af77976f48d5a5cad9ff9a0d20ff)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:09.378294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aecc5f1-0ff0-4063-9250-ba25d8e8d328 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:09.380177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 60/116] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:09 2022] GET /identity/v3/users/5408af77976f48d5a5cad9ff9a0d20ff => generated 289 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:09.380817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecc5f1-0ff0-4063-9250-ba25d8e8d328 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:09.439200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:09.470084 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0129ceb-e4b5-45a9-8fce-9f837598c220 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:09.481794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:09.489683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecc5f1-0ff0-4063-9250-ba25d8e8d328 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:09.506656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af0466-bc4e-402b-ab27-43dd2a4c1f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:09.507688 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af0466-bc4e-402b-ab27-43dd2a4c1f28 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.508185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af0466-bc4e-402b-ab27-43dd2a4c1f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.508693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af0466-bc4e-402b-ab27-43dd2a4c1f28 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.520216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af0466-bc4e-402b-ab27-43dd2a4c1f28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-51af0466-bc4e-402b-ab27-43dd2a4c1f28 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:09.528424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:09.529408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:09.530354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 57/117] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:09.531998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af0466-bc4e-402b-ab27-43dd2a4c1f28 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:09.533689 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92385f70-7ee7-4533-95a1-f84623c6d57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:09.534507 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92385f70-7ee7-4533-95a1-f84623c6d57b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.534798 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92385f70-7ee7-4533-95a1-f84623c6d57b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.535082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92385f70-7ee7-4533-95a1-f84623c6d57b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.548375 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92385f70-7ee7-4533-95a1-f84623c6d57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-92385f70-7ee7-4533-95a1-f84623c6d57b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:09.559022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:09.559904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:09.560811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 61/118] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:09.562001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb59558-9060-4826-b280-d8dcc8403de5 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.562953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb59558-9060-4826-b280-d8dcc8403de5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.563364 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb59558-9060-4826-b280-d8dcc8403de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.584539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5bb59558-9060-4826-b280-d8dcc8403de5 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:09.585640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5bb59558-9060-4826-b280-d8dcc8403de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:09.646944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af0466-bc4e-402b-ab27-43dd2a4c1f28 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:09.661633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 62/119] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:09.662928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb59558-9060-4826-b280-d8dcc8403de5 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:09.696325 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b730d05-d26e-40c9-97a4-d811bed35cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:09.697235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b730d05-d26e-40c9-97a4-d811bed35cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.697517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b730d05-d26e-40c9-97a4-d811bed35cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.697803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b730d05-d26e-40c9-97a4-d811bed35cb7 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.699003 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b730d05-d26e-40c9-97a4-d811bed35cb7 None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:09.700454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b730d05-d26e-40c9-97a4-d811bed35cb7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:09.702321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 58/120] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:09 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:09.703508 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b730d05-d26e-40c9-97a4-d811bed35cb7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:09.707745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:09.767074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:09.784389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb59558-9060-4826-b280-d8dcc8403de5 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:09.819545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b730d05-d26e-40c9-97a4-d811bed35cb7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:09.822230 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e02a15-e8c3-4b4c-ad80-319dee7d271d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:09.822885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e02a15-e8c3-4b4c-ad80-319dee7d271d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.823181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e02a15-e8c3-4b4c-ad80-319dee7d271d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.823467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e02a15-e8c3-4b4c-ad80-319dee7d271d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.854650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410e957-f657-4963-9867-a89088d2b7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:09.855599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410e957-f657-4963-9867-a89088d2b7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:09.856078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410e957-f657-4963-9867-a89088d2b7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:09.856364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410e957-f657-4963-9867-a89088d2b7e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:09.857210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410e957-f657-4963-9867-a89088d2b7e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:09.857530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410e957-f657-4963-9867-a89088d2b7e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:09.858931 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410e957-f657-4963-9867-a89088d2b7e0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:09.872746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 59/121] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:09.872746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410e957-f657-4963-9867-a89088d2b7e0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:09.900292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e02a15-e8c3-4b4c-ad80-319dee7d271d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:09.901921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e02a15-e8c3-4b4c-ad80-319dee7d271d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:09.941373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:09.984632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410e957-f657-4963-9867-a89088d2b7e0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:09.997184 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:10.016185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde59b8-a79a-46da-8479-9e55d1a6ef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:10.016976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde59b8-a79a-46da-8479-9e55d1a6ef1f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:10.017272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde59b8-a79a-46da-8479-9e55d1a6ef1f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:10.017603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde59b8-a79a-46da-8479-9e55d1a6ef1f None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:10.018388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde59b8-a79a-46da-8479-9e55d1a6ef1f None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:10.018705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde59b8-a79a-46da-8479-9e55d1a6ef1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:10.020158 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde59b8-a79a-46da-8479-9e55d1a6ef1f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:10.022962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 63/122] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:10.024087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e02a15-e8c3-4b4c-ad80-319dee7d271d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:10.031661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 60/123] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:10.036873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde59b8-a79a-46da-8479-9e55d1a6ef1f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:10.137052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e02a15-e8c3-4b4c-ad80-319dee7d271d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:10.139951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde59b8-a79a-46da-8479-9e55d1a6ef1f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:10.166868 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:10.176333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbba69-e995-414e-ac53-6971679aa598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:10.176928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8229cd-3776-400c-98ba-79d6f4e464ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:10.177266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbba69-e995-414e-ac53-6971679aa598 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:10.177548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbba69-e995-414e-ac53-6971679aa598 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:10.177726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8229cd-3776-400c-98ba-79d6f4e464ba None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:10.177818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fbba69-e995-414e-ac53-6971679aa598 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490/users/6dcf44525c2549deba53d9ec8dbe03ad/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:10.178014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8229cd-3776-400c-98ba-79d6f4e464ba None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:10.178295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8229cd-3776-400c-98ba-79d6f4e464ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:10.179779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbba69-e995-414e-ac53-6971679aa598 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea37fde41a0c43a790da11fb742b5490, user_id=6dcf44525c2549deba53d9ec8dbe03ad, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:10.181280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fbba69-e995-414e-ac53-6971679aa598 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:10.190535 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8229cd-3776-400c-98ba-79d6f4e464ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:10.196716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 61/124] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:41:10 2022] PUT /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490/users/6dcf44525c2549deba53d9ec8dbe03ad/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:41:10.196716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbba69-e995-414e-ac53-6971679aa598 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-0e8229cd-3776-400c-98ba-79d6f4e464ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:10.209170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:10.210183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:10.211461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 64/125] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:41:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:10.216303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8229cd-3776-400c-98ba-79d6f4e464ba None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:10.280845 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:10.328893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fbba69-e995-414e-ac53-6971679aa598 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:10.362926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8229cd-3776-400c-98ba-79d6f4e464ba None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:10.370481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e61b3-34c8-4a31-9ae1-8d27cd68027f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:10.371257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e61b3-34c8-4a31-9ae1-8d27cd68027f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:10.371551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e61b3-34c8-4a31-9ae1-8d27cd68027f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:10.371834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e61b3-34c8-4a31-9ae1-8d27cd68027f None admin] PATH_INFO: `/v3/users/6dcf44525c2549deba53d9ec8dbe03ad` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:10.373553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e61b3-34c8-4a31-9ae1-8d27cd68027f None admin] RBAC: Authorizing `identity:get_user(user_id=6dcf44525c2549deba53d9ec8dbe03ad)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:10.374617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e61b3-34c8-4a31-9ae1-8d27cd68027f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:10.376454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 62/126] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:10 2022] GET /identity/v3/users/6dcf44525c2549deba53d9ec8dbe03ad => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:10.377101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e61b3-34c8-4a31-9ae1-8d27cd68027f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:10.416949 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c732f1-7f28-4b9d-8821-e9a067c660f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:10.417868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c732f1-7f28-4b9d-8821-e9a067c660f4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:10.418160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c732f1-7f28-4b9d-8821-e9a067c660f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:10.418707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c732f1-7f28-4b9d-8821-e9a067c660f4 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:10.420874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c732f1-7f28-4b9d-8821-e9a067c660f4 None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:10.422914 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c732f1-7f28-4b9d-8821-e9a067c660f4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:10.424941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 65/127] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:10 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:10.425563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c732f1-7f28-4b9d-8821-e9a067c660f4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:10.447459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:10.496925 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e61b3-34c8-4a31-9ae1-8d27cd68027f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:10.533635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c732f1-7f28-4b9d-8821-e9a067c660f4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:10.533829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcfc78-e535-4da5-a27d-a9ec6758f050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:10.534552 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcfc78-e535-4da5-a27d-a9ec6758f050 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:10.534846 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcfc78-e535-4da5-a27d-a9ec6758f050 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:10.535128 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcfc78-e535-4da5-a27d-a9ec6758f050 None admin] PATH_INFO: `/v3/users/6dcf44525c2549deba53d9ec8dbe03ad` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:10.536398 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcfc78-e535-4da5-a27d-a9ec6758f050 None admin] RBAC: Authorizing `identity:get_user(user_id=6dcf44525c2549deba53d9ec8dbe03ad)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:10.537977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcfc78-e535-4da5-a27d-a9ec6758f050 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:10.540316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 63/128] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:10 2022] GET /identity/v3/users/6dcf44525c2549deba53d9ec8dbe03ad => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:10.541035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcfc78-e535-4da5-a27d-a9ec6758f050 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:10.550742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:10.574438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4dc6b-9b3e-4751-932b-9904fee0ffc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:10.575666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4dc6b-9b3e-4751-932b-9904fee0ffc4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:10.576434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4dc6b-9b3e-4751-932b-9904fee0ffc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:10.576885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4dc6b-9b3e-4751-932b-9904fee0ffc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:10.578508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4dc6b-9b3e-4751-932b-9904fee0ffc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:10.579277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4dc6b-9b3e-4751-932b-9904fee0ffc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:10.581096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4dc6b-9b3e-4751-932b-9904fee0ffc4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:10.592157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 66/129] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:10.593266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4dc6b-9b3e-4751-932b-9904fee0ffc4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:10.675064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcfc78-e535-4da5-a27d-a9ec6758f050 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:10.706771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:10.719686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f4bdf-96a9-447e-8aee-2931666a6874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:10.720619 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4dc6b-9b3e-4751-932b-9904fee0ffc4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:10.720995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f4bdf-96a9-447e-8aee-2931666a6874 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:10.721421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f4bdf-96a9-447e-8aee-2931666a6874 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:10.721841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f4bdf-96a9-447e-8aee-2931666a6874 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:10.723389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f4bdf-96a9-447e-8aee-2931666a6874 None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:10.724993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f4bdf-96a9-447e-8aee-2931666a6874 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:10.727029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 64/130] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:10 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:10.727826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f4bdf-96a9-447e-8aee-2931666a6874 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:10.764919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed07be-bdd4-477f-ba2e-ce1fa6f77f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:10.765626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed07be-bdd4-477f-ba2e-ce1fa6f77f77 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:10.765921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed07be-bdd4-477f-ba2e-ce1fa6f77f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:10.766204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed07be-bdd4-477f-ba2e-ce1fa6f77f77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:10.767310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed07be-bdd4-477f-ba2e-ce1fa6f77f77 None admin] RBAC: Adding query filter params (role.id=fe57e073b65947b199b35bd49fc6e30a, scope.project.id=ea37fde41a0c43a790da11fb742b5490, user.id=5408af77976f48d5a5cad9ff9a0d20ff) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:10.767635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed07be-bdd4-477f-ba2e-ce1fa6f77f77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:10.770094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed07be-bdd4-477f-ba2e-ce1fa6f77f77 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:10.780800 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 67/131] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:10 2022] GET /identity/v3/role_assignments?user.id=5408af77976f48d5a5cad9ff9a0d20ff&scope.project.id=ea37fde41a0c43a790da11fb742b5490&role.id=fe57e073b65947b199b35bd49fc6e30a => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:10.782623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed07be-bdd4-477f-ba2e-ce1fa6f77f77 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:10.867817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:10.896835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f4bdf-96a9-447e-8aee-2931666a6874 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:10.965127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ba6c6-0fb2-438a-a1d7-c124bd467522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:10.965912 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ba6c6-0fb2-438a-a1d7-c124bd467522 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:10.966196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ba6c6-0fb2-438a-a1d7-c124bd467522 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:10.966472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ba6c6-0fb2-438a-a1d7-c124bd467522 None admin] PATH_INFO: `/v3/projects/bad446a9e7c7496382a00b02db480740` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:10.968543 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ba6c6-0fb2-438a-a1d7-c124bd467522 None admin] RBAC: Authorizing `identity:get_project(project_id=bad446a9e7c7496382a00b02db480740)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:10.970113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ba6c6-0fb2-438a-a1d7-c124bd467522 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:10.971915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 65/132] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:10 2022] GET /identity/v3/projects/bad446a9e7c7496382a00b02db480740 => generated 312 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:10.976822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ba6c6-0fb2-438a-a1d7-c124bd467522 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:10.994768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed07be-bdd4-477f-ba2e-ce1fa6f77f77 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:11.031912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898ecd7-6d9a-4bf6-a56c-3ad361b2f73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:11.032933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898ecd7-6d9a-4bf6-a56c-3ad361b2f73a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:11.033222 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898ecd7-6d9a-4bf6-a56c-3ad361b2f73a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:11.033505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898ecd7-6d9a-4bf6-a56c-3ad361b2f73a None admin] PATH_INFO: `/v3/users/86b831b5471b48e6bf2bf9ce771279b2` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:11.034836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898ecd7-6d9a-4bf6-a56c-3ad361b2f73a None admin] RBAC: Authorizing `identity:get_user(user_id=86b831b5471b48e6bf2bf9ce771279b2)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:11.036290 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898ecd7-6d9a-4bf6-a56c-3ad361b2f73a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:11.038227 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 68/133] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:10 2022] GET /identity/v3/users/86b831b5471b48e6bf2bf9ce771279b2 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:11.038984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898ecd7-6d9a-4bf6-a56c-3ad361b2f73a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:11.043817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:11.092855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ba6c6-0fb2-438a-a1d7-c124bd467522 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:11.131486 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171969f-0d3b-418e-8055-fa14dda80194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:11.132899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171969f-0d3b-418e-8055-fa14dda80194 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:11.133038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171969f-0d3b-418e-8055-fa14dda80194 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:11.133422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171969f-0d3b-418e-8055-fa14dda80194 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490/users/6dcf44525c2549deba53d9ec8dbe03ad/roles/2d0e0231af5d4e51b80de6bb65e8b02b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:11.135933 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171969f-0d3b-418e-8055-fa14dda80194 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea37fde41a0c43a790da11fb742b5490, user_id=6dcf44525c2549deba53d9ec8dbe03ad, role_id=2d0e0231af5d4e51b80de6bb65e8b02b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:11.137618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171969f-0d3b-418e-8055-fa14dda80194 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:11.153858 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 66/134] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:41:10 2022] PUT /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490/users/6dcf44525c2549deba53d9ec8dbe03ad/roles/2d0e0231af5d4e51b80de6bb65e8b02b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:41:11.162510 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbe46d-0d7c-4bed-b4f4-748704e6b25b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:11.162975 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbe46d-0d7c-4bed-b4f4-748704e6b25b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:11.163373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbe46d-0d7c-4bed-b4f4-748704e6b25b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:11.165446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 67/135] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:11 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:11.166008 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbe46d-0d7c-4bed-b4f4-748704e6b25b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:11.172122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898ecd7-6d9a-4bf6-a56c-3ad361b2f73a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:11.184965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:11.210716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109e452-2c6a-42ad-a372-c6c997c62cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:11.211951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109e452-2c6a-42ad-a372-c6c997c62cde None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:11.212397 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109e452-2c6a-42ad-a372-c6c997c62cde None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:11.212928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109e452-2c6a-42ad-a372-c6c997c62cde None admin] PATH_INFO: `/v3/projects/bad446a9e7c7496382a00b02db480740/users/6dcf44525c2549deba53d9ec8dbe03ad/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:11.219990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109e452-2c6a-42ad-a372-c6c997c62cde None admin] RBAC: Authorizing `identity:create_grant(project_id=bad446a9e7c7496382a00b02db480740, user_id=6dcf44525c2549deba53d9ec8dbe03ad, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:11.223480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109e452-2c6a-42ad-a372-c6c997c62cde None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:11.263308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 69/136] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:41:11 2022] PUT /identity/v3/projects/bad446a9e7c7496382a00b02db480740/users/6dcf44525c2549deba53d9ec8dbe03ad/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:41:11.264921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef77b3-a0d7-46d8-a369-c8dc96fd42f4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:11.265502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef77b3-a0d7-46d8-a369-c8dc96fd42f4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:11.265927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef77b3-a0d7-46d8-a369-c8dc96fd42f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:11.309987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7aef77b3-a0d7-46d8-a369-c8dc96fd42f4 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:11.311680 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7aef77b3-a0d7-46d8-a369-c8dc96fd42f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:11.329748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbe46d-0d7c-4bed-b4f4-748704e6b25b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:11.389876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:11.449352 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c29687-b2da-4126-9088-9e21eee0ee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:11.450270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c29687-b2da-4126-9088-9e21eee0ee6b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:11.450551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c29687-b2da-4126-9088-9e21eee0ee6b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:11.450840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c29687-b2da-4126-9088-9e21eee0ee6b None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:11.457768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c29687-b2da-4126-9088-9e21eee0ee6b None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:11.459187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c29687-b2da-4126-9088-9e21eee0ee6b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:11.461489 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 68/137] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:11 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:11.467298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c29687-b2da-4126-9088-9e21eee0ee6b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:11.483304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 70/138] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:11.508962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef77b3-a0d7-46d8-a369-c8dc96fd42f4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:11.612613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c29687-b2da-4126-9088-9e21eee0ee6b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:11.676585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac9e84-7792-4a1b-b4f3-22c390ba6565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:11.677575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac9e84-7792-4a1b-b4f3-22c390ba6565 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:11.677865 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac9e84-7792-4a1b-b4f3-22c390ba6565 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:11.678142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ac9e84-7792-4a1b-b4f3-22c390ba6565 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490/users/86b831b5471b48e6bf2bf9ce771279b2/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:11.681412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac9e84-7792-4a1b-b4f3-22c390ba6565 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea37fde41a0c43a790da11fb742b5490, user_id=86b831b5471b48e6bf2bf9ce771279b2, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:11.682879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ac9e84-7792-4a1b-b4f3-22c390ba6565 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:11.701617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 69/139] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:41:11 2022] PUT /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490/users/86b831b5471b48e6bf2bf9ce771279b2/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:41:11.711079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:11.717999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef77b3-a0d7-46d8-a369-c8dc96fd42f4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:11.762417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02f9bd-6713-4479-a21c-26e76771b123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:11.763136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02f9bd-6713-4479-a21c-26e76771b123 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:11.763424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02f9bd-6713-4479-a21c-26e76771b123 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:11.763706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02f9bd-6713-4479-a21c-26e76771b123 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:11.769651 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02f9bd-6713-4479-a21c-26e76771b123 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:11.771147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02f9bd-6713-4479-a21c-26e76771b123 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:11.773322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 71/140] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:11.778726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac9e84-7792-4a1b-b4f3-22c390ba6565 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:11.836599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:11.893685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ac9e84-7792-4a1b-b4f3-22c390ba6565 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:11.933525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c9950-51c6-4fea-86b6-03c00953687b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:11.934237 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c9950-51c6-4fea-86b6-03c00953687b None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:11.934528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c9950-51c6-4fea-86b6-03c00953687b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:11.934812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c9950-51c6-4fea-86b6-03c00953687b None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:11.935575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c9950-51c6-4fea-86b6-03c00953687b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:11.937715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c9950-51c6-4fea-86b6-03c00953687b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:11.971697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 70/141] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:41:11 2022] POST /identity/v3/users => generated 283 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:41:12.654427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45b77a-fc34-49aa-98d3-12343ebadb37 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:12.654922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45b77a-fc34-49aa-98d3-12343ebadb37 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:12.655059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45b77a-fc34-49aa-98d3-12343ebadb37 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:12.658552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 72/142] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:12.667951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c0042-127c-4f4f-ba9c-2eb4c65478af None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:12.668857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c0042-127c-4f4f-ba9c-2eb4c65478af None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:12.669357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c0042-127c-4f4f-ba9c-2eb4c65478af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:12.687557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a53c0042-127c-4f4f-ba9c-2eb4c65478af None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:12.689573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a53c0042-127c-4f4f-ba9c-2eb4c65478af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:12.760145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 71/143] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:12.784312 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45b77a-fc34-49aa-98d3-12343ebadb37 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:12.907602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45b77a-fc34-49aa-98d3-12343ebadb37 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:12.951225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc4c48-089f-4afe-ac45-499bcf6058b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:12.952383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc4c48-089f-4afe-ac45-499bcf6058b6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:12.953464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc4c48-089f-4afe-ac45-499bcf6058b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:12.953943 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc4c48-089f-4afe-ac45-499bcf6058b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:12.972174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc4c48-089f-4afe-ac45-499bcf6058b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-7cbc4c48-089f-4afe-ac45-499bcf6058b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:12.980540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:12.981480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:12.982277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:12.982277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:12.982277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:12.982277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:12.982277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:12.982277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:12.982277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:12.982277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:12.982277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:12.982277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:12.982277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:12.982277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:12.985391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 73/144] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:12.991328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c0042-127c-4f4f-ba9c-2eb4c65478af None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:13.034847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:13.104876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c0042-127c-4f4f-ba9c-2eb4c65478af None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:13.141255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f86a4-00cc-4dfa-83a7-e1237740743d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:13.142317 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f86a4-00cc-4dfa-83a7-e1237740743d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:13.142774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f86a4-00cc-4dfa-83a7-e1237740743d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:13.143313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f86a4-00cc-4dfa-83a7-e1237740743d None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:13.144247 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f86a4-00cc-4dfa-83a7-e1237740743d None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:13.144746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f86a4-00cc-4dfa-83a7-e1237740743d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:13.146316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f86a4-00cc-4dfa-83a7-e1237740743d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:13.155395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 72/145] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:13.161490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc4c48-089f-4afe-ac45-499bcf6058b6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:13.189508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:13.287160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc4c48-089f-4afe-ac45-499bcf6058b6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:13.333775 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72e775-6eb7-411f-8842-6a974e2b49ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:13.334980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72e775-6eb7-411f-8842-6a974e2b49ae None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:13.335455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72e775-6eb7-411f-8842-6a974e2b49ae None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:13.335910 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa72e775-6eb7-411f-8842-6a974e2b49ae None admin] PATH_INFO: `/v3/users/6dcf44525c2549deba53d9ec8dbe03ad` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:13.341378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72e775-6eb7-411f-8842-6a974e2b49ae None admin] RBAC: Authorizing `identity:get_user(user_id=6dcf44525c2549deba53d9ec8dbe03ad)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:13.343130 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa72e775-6eb7-411f-8842-6a974e2b49ae None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:13.345602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 74/146] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:13 2022] GET /identity/v3/users/6dcf44525c2549deba53d9ec8dbe03ad => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:13.351224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f86a4-00cc-4dfa-83a7-e1237740743d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:13.463996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:13.470876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f86a4-00cc-4dfa-83a7-e1237740743d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:13.507742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143dd2f2-e163-46d6-a1db-30928014abc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:13.508736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dd2f2-e163-46d6-a1db-30928014abc4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:13.509197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dd2f2-e163-46d6-a1db-30928014abc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:13.509678 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143dd2f2-e163-46d6-a1db-30928014abc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:13.583586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143dd2f2-e163-46d6-a1db-30928014abc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:13.585505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143dd2f2-e163-46d6-a1db-30928014abc4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:13.689270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:13.693269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 73/147] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:13.701777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72e775-6eb7-411f-8842-6a974e2b49ae None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:13.804851 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:13.815456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa72e775-6eb7-411f-8842-6a974e2b49ae None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:13.852859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c075a41-47b3-43a2-8410-a1adc1888edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:13.853975 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c075a41-47b3-43a2-8410-a1adc1888edf None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:13.854416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c075a41-47b3-43a2-8410-a1adc1888edf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:13.854853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c075a41-47b3-43a2-8410-a1adc1888edf None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:13.856200 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c075a41-47b3-43a2-8410-a1adc1888edf None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:13.857833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c075a41-47b3-43a2-8410-a1adc1888edf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:13.859791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 75/148] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:13 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:13.865867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143dd2f2-e163-46d6-a1db-30928014abc4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:13.918135 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229155f-f3fd-4e49-9bba-940357973d73 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:13.918675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229155f-f3fd-4e49-9bba-940357973d73 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:13.919103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229155f-f3fd-4e49-9bba-940357973d73 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:13.920705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 76/149] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:13.926563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe67a22-f30d-4e45-a773-f6d5176578f8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:13.927064 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe67a22-f30d-4e45-a773-f6d5176578f8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:13.927502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe67a22-f30d-4e45-a773-f6d5176578f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:13.953305 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0fe67a22-f30d-4e45-a773-f6d5176578f8 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:13.955005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0fe67a22-f30d-4e45-a773-f6d5176578f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:14.041725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143dd2f2-e163-46d6-a1db-30928014abc4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:14.058839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 77/150] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:14.102060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac143a-da18-4101-82da-a897a348afdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:14.102717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac143a-da18-4101-82da-a897a348afdd None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:14.102959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe67a22-f30d-4e45-a773-f6d5176578f8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:14.103081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac143a-da18-4101-82da-a897a348afdd None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:14.103258 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac143a-da18-4101-82da-a897a348afdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:14.104095 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac143a-da18-4101-82da-a897a348afdd None admin] RBAC: Adding query filter params (role.id=20703235b4c543bc9c51fb7c26da270d, scope.project.id=ea37fde41a0c43a790da11fb742b5490, user.id=6dcf44525c2549deba53d9ec8dbe03ad) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:14.104404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac143a-da18-4101-82da-a897a348afdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:14.106006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac143a-da18-4101-82da-a897a348afdd None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:14.120397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 74/151] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:13 2022] GET /identity/v3/role_assignments?user.id=6dcf44525c2549deba53d9ec8dbe03ad&scope.project.id=ea37fde41a0c43a790da11fb742b5490&role.id=20703235b4c543bc9c51fb7c26da270d => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:14.232823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe67a22-f30d-4e45-a773-f6d5176578f8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:14.237164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:14.274865 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28833e4f-1672-4bf5-9c0b-c7c03fa32195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:14.275672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28833e4f-1672-4bf5-9c0b-c7c03fa32195 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:14.275957 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28833e4f-1672-4bf5-9c0b-c7c03fa32195 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:14.276236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28833e4f-1672-4bf5-9c0b-c7c03fa32195 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:14.288194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28833e4f-1672-4bf5-9c0b-c7c03fa32195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-28833e4f-1672-4bf5-9c0b-c7c03fa32195 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:14.296147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:14.297173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:14.297942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 78/152] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:14.307222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac143a-da18-4101-82da-a897a348afdd None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:14.389935 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:14.439693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac143a-da18-4101-82da-a897a348afdd None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:14.470901 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651c6f6-2fa1-4662-a705-73e992e82db9 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:14.471598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651c6f6-2fa1-4662-a705-73e992e82db9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:14.472072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651c6f6-2fa1-4662-a705-73e992e82db9 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:14.473847 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 79/153] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:41:14.488216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc3c68-65b2-4aa5-81c0-2b0c976d05ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:14.489210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc3c68-65b2-4aa5-81c0-2b0c976d05ee None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:14.489495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc3c68-65b2-4aa5-81c0-2b0c976d05ee None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:14.489771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc3c68-65b2-4aa5-81c0-2b0c976d05ee None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:14.490534 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc3c68-65b2-4aa5-81c0-2b0c976d05ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:14.490844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc3c68-65b2-4aa5-81c0-2b0c976d05ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:14.491153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36d8ed-ebe5-42ae-84dd-41a26d5b1d36 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:14.491635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36d8ed-ebe5-42ae-84dd-41a26d5b1d36 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:14.492116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de36d8ed-ebe5-42ae-84dd-41a26d5b1d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:14.492223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc3c68-65b2-4aa5-81c0-2b0c976d05ee None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:14.506461 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 75/154] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:14.513328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc3c68-65b2-4aa5-81c0-2b0c976d05ee None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:14.525795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-de36d8ed-ebe5-42ae-84dd-41a26d5b1d36 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:14.527342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-de36d8ed-ebe5-42ae-84dd-41a26d5b1d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:14.627975 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 80/155] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:14.629559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc3c68-65b2-4aa5-81c0-2b0c976d05ee None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:14.629887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153a2da-2f79-4f32-9602-b90780629427 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:14.630380 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153a2da-2f79-4f32-9602-b90780629427 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:14.630987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153a2da-2f79-4f32-9602-b90780629427 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:14.632588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 81/156] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:14.639012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45ad4e-92bd-4f61-a886-905221ea83f1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:14.639617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45ad4e-92bd-4f61-a886-905221ea83f1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:14.640179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45ad4e-92bd-4f61-a886-905221ea83f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:14.663575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:14.665675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3d45ad4e-92bd-4f61-a886-905221ea83f1 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:14.667185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3d45ad4e-92bd-4f61-a886-905221ea83f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:14.689444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cb10a-89b3-47f6-84c3-6ee488d2606c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:14.690536 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cb10a-89b3-47f6-84c3-6ee488d2606c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:14.690892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cb10a-89b3-47f6-84c3-6ee488d2606c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:14.691237 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cb10a-89b3-47f6-84c3-6ee488d2606c None admin] PATH_INFO: `/v3/users/5408af77976f48d5a5cad9ff9a0d20ff` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:14.695121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cb10a-89b3-47f6-84c3-6ee488d2606c None admin] RBAC: Authorizing `identity:get_user(user_id=5408af77976f48d5a5cad9ff9a0d20ff)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:14.699610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cb10a-89b3-47f6-84c3-6ee488d2606c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:14.699610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 76/157] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:14 2022] GET /identity/v3/users/5408af77976f48d5a5cad9ff9a0d20ff => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:14.705761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cb10a-89b3-47f6-84c3-6ee488d2606c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:14.759843 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 82/158] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:14.761460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45ad4e-92bd-4f61-a886-905221ea83f1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:14.774387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:14.849777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cb10a-89b3-47f6-84c3-6ee488d2606c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:14.873449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45ad4e-92bd-4f61-a886-905221ea83f1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:14.890301 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b3417-1f13-4a45-8359-46c938ebfc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:14.891480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b3417-1f13-4a45-8359-46c938ebfc59 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:14.892020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b3417-1f13-4a45-8359-46c938ebfc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:14.892443 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b3417-1f13-4a45-8359-46c938ebfc59 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:14.896326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b3417-1f13-4a45-8359-46c938ebfc59 None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:14.898063 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b3417-1f13-4a45-8359-46c938ebfc59 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:14.900331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 77/159] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:14 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:14.905583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b3417-1f13-4a45-8359-46c938ebfc59 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:14.911839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d3bb5-7cd1-482d-a9ba-36b4dcd69eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:14.912818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d3bb5-7cd1-482d-a9ba-36b4dcd69eaa None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:14.913262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d3bb5-7cd1-482d-a9ba-36b4dcd69eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:14.913707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d3bb5-7cd1-482d-a9ba-36b4dcd69eaa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:14.925980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d3bb5-7cd1-482d-a9ba-36b4dcd69eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-362d3bb5-7cd1-482d-a9ba-36b4dcd69eaa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:14.934642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:14.935681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:14.936476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:14.936476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:14.936476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:14.936476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:14.936476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:14.936476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:14.936476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:14.936476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:14.936476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:14.936476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:14.936476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:14.936476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:14.938707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 83/160] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:14.939705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d3bb5-7cd1-482d-a9ba-36b4dcd69eaa None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:15.009044 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:15.022256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:15.053772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d3bb5-7cd1-482d-a9ba-36b4dcd69eaa None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:15.072051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b3417-1f13-4a45-8359-46c938ebfc59 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:15.088984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454c64a-95be-40ce-a35c-1e7fb95e74b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:15.090229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454c64a-95be-40ce-a35c-1e7fb95e74b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:15.090717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454c64a-95be-40ce-a35c-1e7fb95e74b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:15.091196 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454c64a-95be-40ce-a35c-1e7fb95e74b6 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490/users/5408af77976f48d5a5cad9ff9a0d20ff/roles/fe57e073b65947b199b35bd49fc6e30a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:15.093998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454c64a-95be-40ce-a35c-1e7fb95e74b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea37fde41a0c43a790da11fb742b5490, user_id=5408af77976f48d5a5cad9ff9a0d20ff, role_id=fe57e073b65947b199b35bd49fc6e30a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:15.095684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454c64a-95be-40ce-a35c-1e7fb95e74b6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:15.115550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 84/161] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:41:14 2022] PUT /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490/users/5408af77976f48d5a5cad9ff9a0d20ff/roles/fe57e073b65947b199b35bd49fc6e30a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:41:15.117790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454c64a-95be-40ce-a35c-1e7fb95e74b6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:15.150187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc7173-cfec-4bf4-8363-16673af82a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:15.151333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc7173-cfec-4bf4-8363-16673af82a60 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:15.151897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc7173-cfec-4bf4-8363-16673af82a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:15.152378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc7173-cfec-4bf4-8363-16673af82a60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:15.178803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc7173-cfec-4bf4-8363-16673af82a60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-aefc7173-cfec-4bf4-8363-16673af82a60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:15.194059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:15.194988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:15.195844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:15.195844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:15.195844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:15.195844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:15.195844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:15.195844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:15.195844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:15.195844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:15.195844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:15.195844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:15.195844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:15.195844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:15.199700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 78/162] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:15.210049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc7173-cfec-4bf4-8363-16673af82a60 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:15.365080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454c64a-95be-40ce-a35c-1e7fb95e74b6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:15.415282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:15.433261 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c8717-e423-4856-a62d-d9640ab7f1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:15.433963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c8717-e423-4856-a62d-d9640ab7f1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:15.434256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c8717-e423-4856-a62d-d9640ab7f1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:15.434539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c8717-e423-4856-a62d-d9640ab7f1f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:15.435310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c8717-e423-4856-a62d-d9640ab7f1f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:15.435626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c8717-e423-4856-a62d-d9640ab7f1f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:15.441561 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c8717-e423-4856-a62d-d9640ab7f1f7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:15.457401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 85/163] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:15.463442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c8717-e423-4856-a62d-d9640ab7f1f7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:15.471583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc7173-cfec-4bf4-8363-16673af82a60 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:15.513057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:15.536696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0bbba-2389-4f7e-8587-33d9d165963a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:15.537382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0bbba-2389-4f7e-8587-33d9d165963a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:15.537712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0bbba-2389-4f7e-8587-33d9d165963a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:15.537995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0bbba-2389-4f7e-8587-33d9d165963a None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:15.538757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0bbba-2389-4f7e-8587-33d9d165963a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:15.539066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0bbba-2389-4f7e-8587-33d9d165963a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:15.540460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0bbba-2389-4f7e-8587-33d9d165963a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:15.558772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 79/164] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:15.563600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0bbba-2389-4f7e-8587-33d9d165963a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:15.660621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c8717-e423-4856-a62d-d9640ab7f1f7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:15.729300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9b4eb-48e0-4d5e-a5bd-220a97ac7241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:15.731576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9b4eb-48e0-4d5e-a5bd-220a97ac7241 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:15.732791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9b4eb-48e0-4d5e-a5bd-220a97ac7241 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:15.733231 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9b4eb-48e0-4d5e-a5bd-220a97ac7241 None admin] PATH_INFO: `/v3/users/6dcf44525c2549deba53d9ec8dbe03ad` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:15.737322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9b4eb-48e0-4d5e-a5bd-220a97ac7241 None admin] RBAC: Authorizing `identity:get_user(user_id=6dcf44525c2549deba53d9ec8dbe03ad)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:15.738945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9b4eb-48e0-4d5e-a5bd-220a97ac7241 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:15.741898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 86/165] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:15 2022] GET /identity/v3/users/6dcf44525c2549deba53d9ec8dbe03ad => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:15.743499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a5348-e855-4590-974c-9c8f73e2dec4 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:15.745180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a5348-e855-4590-974c-9c8f73e2dec4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:15.745479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a5348-e855-4590-974c-9c8f73e2dec4 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:15.747323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 87/166] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:15.748102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a5348-e855-4590-974c-9c8f73e2dec4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:15.763918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0bbba-2389-4f7e-8587-33d9d165963a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:15.807184 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a57213-8f9b-4055-bd86-46d7440865c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:15.808074 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a57213-8f9b-4055-bd86-46d7440865c2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:15.808362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a57213-8f9b-4055-bd86-46d7440865c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:15.808643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a57213-8f9b-4055-bd86-46d7440865c2 None admin] PATH_INFO: `/v3/users/6dcf44525c2549deba53d9ec8dbe03ad` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:15.810308 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a57213-8f9b-4055-bd86-46d7440865c2 None admin] RBAC: Authorizing `identity:get_user(user_id=6dcf44525c2549deba53d9ec8dbe03ad)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:15.811481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a57213-8f9b-4055-bd86-46d7440865c2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:15.816819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 80/167] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:15 2022] GET /identity/v3/users/6dcf44525c2549deba53d9ec8dbe03ad => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:15.818478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78bc05-0edf-4bcf-8dc5-40cf8ef32083 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:15.818478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78bc05-0edf-4bcf-8dc5-40cf8ef32083 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:15.818753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78bc05-0edf-4bcf-8dc5-40cf8ef32083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:15.834300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:15.843487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bd78bc05-0edf-4bcf-8dc5-40cf8ef32083 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:15.844943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bd78bc05-0edf-4bcf-8dc5-40cf8ef32083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:15.886742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a5348-e855-4590-974c-9c8f73e2dec4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:15.897510 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:15.924044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c4b66-2764-481c-af62-d6177803607d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:15.924967 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c4b66-2764-481c-af62-d6177803607d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:15.925092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 81/168] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:15.925505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c4b66-2764-481c-af62-d6177803607d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:15.925922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c4b66-2764-481c-af62-d6177803607d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:15.929502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78bc05-0edf-4bcf-8dc5-40cf8ef32083 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:15.997016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c4b66-2764-481c-af62-d6177803607d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:15.998717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c4b66-2764-481c-af62-d6177803607d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:16.058233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:16.066050 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78bc05-0edf-4bcf-8dc5-40cf8ef32083 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:16.101863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 88/169] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:16.103277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c4b66-2764-481c-af62-d6177803607d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:16.103934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cf7b0-a254-4e3f-99f4-498254a5fa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:16.104643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cf7b0-a254-4e3f-99f4-498254a5fa84 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:16.105009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cf7b0-a254-4e3f-99f4-498254a5fa84 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:16.105292 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447cf7b0-a254-4e3f-99f4-498254a5fa84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:16.157668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:16.185196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cf7b0-a254-4e3f-99f4-498254a5fa84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:16.186619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447cf7b0-a254-4e3f-99f4-498254a5fa84 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:16.210257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c4b66-2764-481c-af62-d6177803607d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:16.245631 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd320b-ef94-4f7c-9c9f-52457b192379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:16.246707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd320b-ef94-4f7c-9c9f-52457b192379 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:16.247151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd320b-ef94-4f7c-9c9f-52457b192379 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:16.247572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd320b-ef94-4f7c-9c9f-52457b192379 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:16.258789 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd320b-ef94-4f7c-9c9f-52457b192379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-a6cd320b-ef94-4f7c-9c9f-52457b192379 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:16.266822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:16.267872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:16.268835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:16.268835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:16.268835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:16.268835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:16.268835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:16.268835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:16.268835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:16.268835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:16.268835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:16.268835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:16.268835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:16.268835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:16.270746 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 89/170] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:41:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:16.271657 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd320b-ef94-4f7c-9c9f-52457b192379 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:16.297873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:16.349278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 82/171] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:16.350143 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cf7b0-a254-4e3f-99f4-498254a5fa84 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:16.381295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd320b-ef94-4f7c-9c9f-52457b192379 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:16.416830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44377e4-01da-4e9d-80e4-7896a6506c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:16.417945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44377e4-01da-4e9d-80e4-7896a6506c2e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:16.418377 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44377e4-01da-4e9d-80e4-7896a6506c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:16.418826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44377e4-01da-4e9d-80e4-7896a6506c2e None admin] PATH_INFO: `/v3/projects/bad446a9e7c7496382a00b02db480740` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:16.420177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44377e4-01da-4e9d-80e4-7896a6506c2e None admin] RBAC: Authorizing `identity:get_project(project_id=bad446a9e7c7496382a00b02db480740)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:16.422405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44377e4-01da-4e9d-80e4-7896a6506c2e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:16.424312 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 90/172] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:16 2022] GET /identity/v3/projects/bad446a9e7c7496382a00b02db480740 => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:16.425296 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44377e4-01da-4e9d-80e4-7896a6506c2e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:16.481052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:16.515350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:16.527874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44377e4-01da-4e9d-80e4-7896a6506c2e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:16.528750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447cf7b0-a254-4e3f-99f4-498254a5fa84 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:16.564025 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0ffa8-1517-4793-a495-06e5a02d104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:16.565288 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0ffa8-1517-4793-a495-06e5a02d104a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:16.572694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0ffa8-1517-4793-a495-06e5a02d104a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:16.573287 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0ffa8-1517-4793-a495-06e5a02d104a None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:16.574828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0ffa8-1517-4793-a495-06e5a02d104a None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:16.576478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0ffa8-1517-4793-a495-06e5a02d104a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:16.578794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 91/173] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:16 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:16.579756 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0ffa8-1517-4793-a495-06e5a02d104a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:16.605051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff969621-e5b9-471a-81aa-6de10b357cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:16.605803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff969621-e5b9-471a-81aa-6de10b357cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:16.606253 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff969621-e5b9-471a-81aa-6de10b357cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:16.606376 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff969621-e5b9-471a-81aa-6de10b357cc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:16.607269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff969621-e5b9-471a-81aa-6de10b357cc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:16.607523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff969621-e5b9-471a-81aa-6de10b357cc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:16.609563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff969621-e5b9-471a-81aa-6de10b357cc8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:16.623860 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 83/174] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:16.633587 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff969621-e5b9-471a-81aa-6de10b357cc8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:16.731627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0ffa8-1517-4793-a495-06e5a02d104a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:16.776829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff969621-e5b9-471a-81aa-6de10b357cc8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:16.791310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971297b2-1575-4fe9-835e-530f49476b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:16.792331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971297b2-1575-4fe9-835e-530f49476b34 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:16.801081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971297b2-1575-4fe9-835e-530f49476b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:16.801670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971297b2-1575-4fe9-835e-530f49476b34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:16.802773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971297b2-1575-4fe9-835e-530f49476b34 None admin] RBAC: Adding query filter params (role.id=20703235b4c543bc9c51fb7c26da270d, scope.project.id=bad446a9e7c7496382a00b02db480740, user.id=6dcf44525c2549deba53d9ec8dbe03ad) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:16.803304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971297b2-1575-4fe9-835e-530f49476b34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:16.805013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971297b2-1575-4fe9-835e-530f49476b34 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:16.810271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029f0d5-0ad8-4b17-b4c7-8950043379a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:16.810944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029f0d5-0ad8-4b17-b4c7-8950043379a8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:16.811817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029f0d5-0ad8-4b17-b4c7-8950043379a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:16.811817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f029f0d5-0ad8-4b17-b4c7-8950043379a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:16.812965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029f0d5-0ad8-4b17-b4c7-8950043379a8 None admin] RBAC: Adding query filter params (role.id=2d0e0231af5d4e51b80de6bb65e8b02b, scope.project.id=ea37fde41a0c43a790da11fb742b5490, user.id=6dcf44525c2549deba53d9ec8dbe03ad) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:16.816694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029f0d5-0ad8-4b17-b4c7-8950043379a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:16.816694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f029f0d5-0ad8-4b17-b4c7-8950043379a8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:16.816874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 92/175] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:16 2022] GET /identity/v3/role_assignments?user.id=6dcf44525c2549deba53d9ec8dbe03ad&scope.project.id=bad446a9e7c7496382a00b02db480740&role.id=20703235b4c543bc9c51fb7c26da270d => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:16.816874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971297b2-1575-4fe9-835e-530f49476b34 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:16.821877 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 84/176] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:16 2022] GET /identity/v3/role_assignments?user.id=6dcf44525c2549deba53d9ec8dbe03ad&scope.project.id=ea37fde41a0c43a790da11fb742b5490&role.id=2d0e0231af5d4e51b80de6bb65e8b02b => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:16.858517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:16.967330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971297b2-1575-4fe9-835e-530f49476b34 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:17.006601 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa310f0-de08-420a-98b6-f74bcfc85271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:17.007892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa310f0-de08-420a-98b6-f74bcfc85271 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.008375 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa310f0-de08-420a-98b6-f74bcfc85271 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.008883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa310f0-de08-420a-98b6-f74bcfc85271 None admin] PATH_INFO: `/v3/users/86b831b5471b48e6bf2bf9ce771279b2` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.010316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa310f0-de08-420a-98b6-f74bcfc85271 None admin] RBAC: Authorizing `identity:get_user(user_id=86b831b5471b48e6bf2bf9ce771279b2)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:17.012100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa310f0-de08-420a-98b6-f74bcfc85271 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:17.014327 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 93/177] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:16 2022] GET /identity/v3/users/86b831b5471b48e6bf2bf9ce771279b2 => generated 247 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:17.025130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029f0d5-0ad8-4b17-b4c7-8950043379a8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:17.029214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:17.147970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f029f0d5-0ad8-4b17-b4c7-8950043379a8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:17.191409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2388992-735a-4b1c-a0ad-6785b81b85f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:17.192083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2388992-735a-4b1c-a0ad-6785b81b85f7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.192372 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2388992-735a-4b1c-a0ad-6785b81b85f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.192654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2388992-735a-4b1c-a0ad-6785b81b85f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.214181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a4770-e5d3-48d4-92e1-6c9a5245bb90 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.214819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a4770-e5d3-48d4-92e1-6c9a5245bb90 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.215301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a4770-e5d3-48d4-92e1-6c9a5245bb90 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.217884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 94/178] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:17.224011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726380b0-4757-449c-9740-56c71873985b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.224586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726380b0-4757-449c-9740-56c71873985b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.225117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726380b0-4757-449c-9740-56c71873985b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.247179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-726380b0-4757-449c-9740-56c71873985b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:17.249020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-726380b0-4757-449c-9740-56c71873985b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:17.267504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2388992-735a-4b1c-a0ad-6785b81b85f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:17.268951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2388992-735a-4b1c-a0ad-6785b81b85f7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:17.272015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:17.331182 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 95/179] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:17.359822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726380b0-4757-449c-9740-56c71873985b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:17.381584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 85/180] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:17.388111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2388992-735a-4b1c-a0ad-6785b81b85f7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:17.405065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:17.479898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726380b0-4757-449c-9740-56c71873985b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:17.496201 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2388992-735a-4b1c-a0ad-6785b81b85f7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:17.498618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:17.523031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683de36-970e-40a9-84f7-4a1dfb4cd63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:17.524181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683de36-970e-40a9-84f7-4a1dfb4cd63c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.524751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683de36-970e-40a9-84f7-4a1dfb4cd63c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.525231 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683de36-970e-40a9-84f7-4a1dfb4cd63c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.538624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278e986-6933-4692-afae-93434555f409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:17.539502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278e986-6933-4692-afae-93434555f409 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.539793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278e986-6933-4692-afae-93434555f409 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.539896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683de36-970e-40a9-84f7-4a1dfb4cd63c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:17.540073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278e986-6933-4692-afae-93434555f409 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.541350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278e986-6933-4692-afae-93434555f409 None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:17.542821 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278e986-6933-4692-afae-93434555f409 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:17.544545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 86/181] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:17 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-9683de36-970e-40a9-84f7-4a1dfb4cd63c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:17.549908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:17.550819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:17.551645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:17.551645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:17.551645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:17.551645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:17.551645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:17.551645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:17.551645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:17.551645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:17.551645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:17.551645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:17.551645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:17.551645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:17.552249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278e986-6933-4692-afae-93434555f409 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:17.554414 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 96/182] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:41:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:41:17.560815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683de36-970e-40a9-84f7-4a1dfb4cd63c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:17.669051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:17.677190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278e986-6933-4692-afae-93434555f409 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:17.689552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683de36-970e-40a9-84f7-4a1dfb4cd63c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:17.722349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd714e7-1965-425f-bd04-8ad6c9f810a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:17.723329 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd714e7-1965-425f-bd04-8ad6c9f810a9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.723718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd714e7-1965-425f-bd04-8ad6c9f810a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.724392 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd714e7-1965-425f-bd04-8ad6c9f810a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.725636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd714e7-1965-425f-bd04-8ad6c9f810a9 None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=ea37fde41a0c43a790da11fb742b5490, user.id=86b831b5471b48e6bf2bf9ce771279b2) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:17.726053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd714e7-1965-425f-bd04-8ad6c9f810a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:17.727873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd714e7-1965-425f-bd04-8ad6c9f810a9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:17.730088 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137b0d0-8a64-4e01-a6b3-d7bc7785dd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:17.731038 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137b0d0-8a64-4e01-a6b3-d7bc7785dd93 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.731537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137b0d0-8a64-4e01-a6b3-d7bc7785dd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.732016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1137b0d0-8a64-4e01-a6b3-d7bc7785dd93 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.733028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137b0d0-8a64-4e01-a6b3-d7bc7785dd93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:17.733545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137b0d0-8a64-4e01-a6b3-d7bc7785dd93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:17.734972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 87/183] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:17 2022] GET /identity/v3/role_assignments?user.id=86b831b5471b48e6bf2bf9ce771279b2&scope.project.id=ea37fde41a0c43a790da11fb742b5490&role.id=569707378794437b93d168fc8bb9cbc3 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:17.735875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1137b0d0-8a64-4e01-a6b3-d7bc7785dd93 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:17.736343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5db71-a4ab-4eda-b2a5-361ea70e7385 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.736633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5db71-a4ab-4eda-b2a5-361ea70e7385 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.736983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5db71-a4ab-4eda-b2a5-361ea70e7385 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.740768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 88/184] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:17.744526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc08e5e-0587-4b17-8b4b-f69fcccc1f1a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.744860 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc08e5e-0587-4b17-8b4b-f69fcccc1f1a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.745141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc08e5e-0587-4b17-8b4b-f69fcccc1f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.755249 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 97/185] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:17.761129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137b0d0-8a64-4e01-a6b3-d7bc7785dd93 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:17.768317 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8fc08e5e-0587-4b17-8b4b-f69fcccc1f1a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:17.769944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8fc08e5e-0587-4b17-8b4b-f69fcccc1f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:17.809483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:17.857821 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 89/186] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:17.868696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415221a7-5883-4eba-b245-08e01c27744b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.868696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415221a7-5883-4eba-b245-08e01c27744b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.868696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415221a7-5883-4eba-b245-08e01c27744b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.868696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 90/187] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:17.868696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a17bc4-13e7-4fe8-8467-f01e979e10c4 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.868696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a17bc4-13e7-4fe8-8467-f01e979e10c4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.869270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a17bc4-13e7-4fe8-8467-f01e979e10c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.899397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-45a17bc4-13e7-4fe8-8467-f01e979e10c4 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:17.901898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-45a17bc4-13e7-4fe8-8467-f01e979e10c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:17.917239 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1137b0d0-8a64-4e01-a6b3-d7bc7785dd93 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:17.962398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ec689-929d-412d-b389-ec90b5472454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:17.964436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ec689-929d-412d-b389-ec90b5472454 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:17.965015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ec689-929d-412d-b389-ec90b5472454 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:17.965494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ec689-929d-412d-b389-ec90b5472454 None admin] PATH_INFO: `/v3/users/f859f8bbfa2b425194813beaee73cecf` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:17.982166 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:17.984624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ec689-929d-412d-b389-ec90b5472454 None admin] RBAC: Authorizing `identity:get_user(user_id=f859f8bbfa2b425194813beaee73cecf)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:17.985672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2ec689-929d-412d-b389-ec90b5472454 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:17.991974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 98/188] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:17 2022] GET /identity/v3/users/f859f8bbfa2b425194813beaee73cecf => generated 283 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:17.992855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ec689-929d-412d-b389-ec90b5472454 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:17.995767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 91/189] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:18.002557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a17bc4-13e7-4fe8-8467-f01e979e10c4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:18.106535 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:18.113772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a17bc4-13e7-4fe8-8467-f01e979e10c4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:18.117345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2ec689-929d-412d-b389-ec90b5472454 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:18.151109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ce421-eff4-496a-a1ba-4e413eb53263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:18.151761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ce421-eff4-496a-a1ba-4e413eb53263 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:18.152049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ce421-eff4-496a-a1ba-4e413eb53263 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:18.152330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142ce421-eff4-496a-a1ba-4e413eb53263 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:18.154069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c880fd-e8bb-484c-b72a-9125a288ffd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:18.154887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c880fd-e8bb-484c-b72a-9125a288ffd3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:18.155148 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c880fd-e8bb-484c-b72a-9125a288ffd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:18.155428 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c880fd-e8bb-484c-b72a-9125a288ffd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:18.156931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c880fd-e8bb-484c-b72a-9125a288ffd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:18.158381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c880fd-e8bb-484c-b72a-9125a288ffd3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:18.164637 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 99/190] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:18.165400 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c880fd-e8bb-484c-b72a-9125a288ffd3 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:18.238909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:18.247378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ce421-eff4-496a-a1ba-4e413eb53263 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:18.248967 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142ce421-eff4-496a-a1ba-4e413eb53263 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:18.277065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c880fd-e8bb-484c-b72a-9125a288ffd3 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:18.314404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de372546-28a8-4c79-b9b2-7e9b7aaef3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:18.315362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de372546-28a8-4c79-b9b2-7e9b7aaef3be None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:18.315970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de372546-28a8-4c79-b9b2-7e9b7aaef3be None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:18.316466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de372546-28a8-4c79-b9b2-7e9b7aaef3be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:18.329193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de372546-28a8-4c79-b9b2-7e9b7aaef3be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-de372546-28a8-4c79-b9b2-7e9b7aaef3be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:18.337373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:18.338316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 100/191] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:18.345129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de372546-28a8-4c79-b9b2-7e9b7aaef3be None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:18.361808 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:18.369514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 92/192] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:18.370246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ce421-eff4-496a-a1ba-4e413eb53263 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:18.457076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de372546-28a8-4c79-b9b2-7e9b7aaef3be None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:18.471333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142ce421-eff4-496a-a1ba-4e413eb53263 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:18.482004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:18.491176 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8492b-a7e9-46b0-a438-1471d6cecf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:18.491855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8492b-a7e9-46b0-a438-1471d6cecf58 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:18.492137 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8492b-a7e9-46b0-a438-1471d6cecf58 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:18.492419 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8492b-a7e9-46b0-a438-1471d6cecf58 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:18.493165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8492b-a7e9-46b0-a438-1471d6cecf58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:18.494607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e8492b-a7e9-46b0-a438-1471d6cecf58 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:18.505606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c7257-aec5-43a9-8fec-40143773c773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:18.506492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c7257-aec5-43a9-8fec-40143773c773 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:18.506886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c7257-aec5-43a9-8fec-40143773c773 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:18.507262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c7257-aec5-43a9-8fec-40143773c773 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:18.508344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c7257-aec5-43a9-8fec-40143773c773 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:18.508712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c7257-aec5-43a9-8fec-40143773c773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:18.510458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c7257-aec5-43a9-8fec-40143773c773 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:18.514792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 101/193] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:41:18 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:41:18.514792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8492b-a7e9-46b0-a438-1471d6cecf58 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:18.529011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 93/194] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:18.529011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f40905-3474-45fe-8554-a116a7c5cb66 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:18.529011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f40905-3474-45fe-8554-a116a7c5cb66 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:18.529011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f40905-3474-45fe-8554-a116a7c5cb66 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:18.529011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 94/195] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:18.529011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f40905-3474-45fe-8554-a116a7c5cb66 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:18.597331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:18.640359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8492b-a7e9-46b0-a438-1471d6cecf58 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:18.649358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f40905-3474-45fe-8554-a116a7c5cb66 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:18.686695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce803f-d7f2-49ed-9bb5-eb4cffbe26f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:18.688100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce803f-d7f2-49ed-9bb5-eb4cffbe26f9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:18.688383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce803f-d7f2-49ed-9bb5-eb4cffbe26f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:18.688655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce803f-d7f2-49ed-9bb5-eb4cffbe26f9 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:18.689337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013368c3-dc6c-48be-ac2e-43ad7520e8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:18.689870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce803f-d7f2-49ed-9bb5-eb4cffbe26f9 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:18.690204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013368c3-dc6c-48be-ac2e-43ad7520e8bc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:18.691487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013368c3-dc6c-48be-ac2e-43ad7520e8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:18.691487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013368c3-dc6c-48be-ac2e-43ad7520e8bc None admin] PATH_INFO: `/v3/users/f859f8bbfa2b425194813beaee73cecf` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:18.691657 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce803f-d7f2-49ed-9bb5-eb4cffbe26f9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:18.696250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013368c3-dc6c-48be-ac2e-43ad7520e8bc None admin] RBAC: Authorizing `identity:get_user(user_id=f859f8bbfa2b425194813beaee73cecf)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:18.696403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 102/196] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:18 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:18.696403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75767a3-ae8a-4ad7-8e69-4cdd605cdd35 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:18.696537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013368c3-dc6c-48be-ac2e-43ad7520e8bc None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:18.696625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75767a3-ae8a-4ad7-8e69-4cdd605cdd35 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:18.696625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75767a3-ae8a-4ad7-8e69-4cdd605cdd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:18.696786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 95/197] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:18 2022] GET /identity/v3/users/f859f8bbfa2b425194813beaee73cecf => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:18.701584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013368c3-dc6c-48be-ac2e-43ad7520e8bc None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:18.713851 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f75767a3-ae8a-4ad7-8e69-4cdd605cdd35 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:18.715074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f75767a3-ae8a-4ad7-8e69-4cdd605cdd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:18.782293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:18.816162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 103/198] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:18.817065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75767a3-ae8a-4ad7-8e69-4cdd605cdd35 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:18.838421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013368c3-dc6c-48be-ac2e-43ad7520e8bc None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:18.872821 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:18.888090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419d094-9a65-41fc-82e7-39eb07892485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:18.889355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d094-9a65-41fc-82e7-39eb07892485 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:18.889743 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d094-9a65-41fc-82e7-39eb07892485 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:18.891186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419d094-9a65-41fc-82e7-39eb07892485 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:18.948318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75767a3-ae8a-4ad7-8e69-4cdd605cdd35 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:18.991037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419d094-9a65-41fc-82e7-39eb07892485 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:18.992534 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7419d094-9a65-41fc-82e7-39eb07892485 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:18.995905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75818e-6a9d-494e-a949-610287246d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:18.996695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75818e-6a9d-494e-a949-610287246d78 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:18.997098 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75818e-6a9d-494e-a949-610287246d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:18.997477 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f75818e-6a9d-494e-a949-610287246d78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:18.998421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75818e-6a9d-494e-a949-610287246d78 None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=fcf4e9bfa6db4a5686e55857c6019793, user.id=f859f8bbfa2b425194813beaee73cecf) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:18.998839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75818e-6a9d-494e-a949-610287246d78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:19.001005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f75818e-6a9d-494e-a949-610287246d78 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:19.007643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 104/199] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:18 2022] GET /identity/v3/role_assignments?user.id=f859f8bbfa2b425194813beaee73cecf&scope.project.id=fcf4e9bfa6db4a5686e55857c6019793&role.id=569707378794437b93d168fc8bb9cbc3 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:19.013274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75818e-6a9d-494e-a949-610287246d78 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:19.120369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:19.190738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 96/200] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 11:41:19.196702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419d094-9a65-41fc-82e7-39eb07892485 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:19.242578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f75818e-6a9d-494e-a949-610287246d78 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:19.244501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:19.306518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8db3c-677c-43d4-b3c7-355aba1f977f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:19.307328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8db3c-677c-43d4-b3c7-355aba1f977f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:19.307610 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8db3c-677c-43d4-b3c7-355aba1f977f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:19.307887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8db3c-677c-43d4-b3c7-355aba1f977f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:19.331848 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8db3c-677c-43d4-b3c7-355aba1f977f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-b0d8db3c-677c-43d4-b3c7-355aba1f977f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:19.340498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:19.341445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:19.342265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:19.342265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:19.342265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:19.342265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:19.342265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:19.342265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:19.342265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:19.342265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:19.342265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:19.342265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:19.342265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:19.342265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:19.342956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 105/201] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:19.348391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8db3c-677c-43d4-b3c7-355aba1f977f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:19.380132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419d094-9a65-41fc-82e7-39eb07892485 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:19.448751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a8561-47e3-4fa6-965a-b855f5539399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:19.451366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a8561-47e3-4fa6-965a-b855f5539399 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:19.451662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a8561-47e3-4fa6-965a-b855f5539399 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:19.451943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045a8561-47e3-4fa6-965a-b855f5539399 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:19.456285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a8561-47e3-4fa6-965a-b855f5539399 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:19.458601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045a8561-47e3-4fa6-965a-b855f5539399 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:19.467567 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 97/202] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:19 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:19.472617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a8561-47e3-4fa6-965a-b855f5539399 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:19.566789 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8db3c-677c-43d4-b3c7-355aba1f977f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:19.574417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:19.587112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:19.595501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045a8561-47e3-4fa6-965a-b855f5539399 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:19.644927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320fe71-597b-4509-9367-a062d0d9c399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:19.645130 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d255f94-a27b-461d-a65c-3d44d39f0ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:19.645594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320fe71-597b-4509-9367-a062d0d9c399 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:19.645887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320fe71-597b-4509-9367-a062d0d9c399 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:19.646173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320fe71-597b-4509-9367-a062d0d9c399 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:19.646696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d255f94-a27b-461d-a65c-3d44d39f0ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:19.647030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320fe71-597b-4509-9367-a062d0d9c399 None admin] RBAC: Adding query filter params (role.id=2d0e0231af5d4e51b80de6bb65e8b02b, scope.project.id=fcf4e9bfa6db4a5686e55857c6019793, user.id=f859f8bbfa2b425194813beaee73cecf) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:19.647349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320fe71-597b-4509-9367-a062d0d9c399 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:19.647594 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d255f94-a27b-461d-a65c-3d44d39f0ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:19.648061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d255f94-a27b-461d-a65c-3d44d39f0ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:19.648829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d320fe71-597b-4509-9367-a062d0d9c399 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:19.649247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d255f94-a27b-461d-a65c-3d44d39f0ba1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:19.649748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d255f94-a27b-461d-a65c-3d44d39f0ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:19.651420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d255f94-a27b-461d-a65c-3d44d39f0ba1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:19.661959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 106/203] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:19.666530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 98/204] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:19 2022] GET /identity/v3/role_assignments?user.id=f859f8bbfa2b425194813beaee73cecf&scope.project.id=fcf4e9bfa6db4a5686e55857c6019793&role.id=2d0e0231af5d4e51b80de6bb65e8b02b => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:19.667840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d255f94-a27b-461d-a65c-3d44d39f0ba1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:19.791505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d255f94-a27b-461d-a65c-3d44d39f0ba1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:19.830345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b12ac-1767-4808-97e7-33a696d19177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:19.831249 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b12ac-1767-4808-97e7-33a696d19177 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:19.831535 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b12ac-1767-4808-97e7-33a696d19177 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:19.831816 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b12ac-1767-4808-97e7-33a696d19177 None admin] PATH_INFO: `/v3/users/5408af77976f48d5a5cad9ff9a0d20ff` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:19.833079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b12ac-1767-4808-97e7-33a696d19177 None admin] RBAC: Authorizing `identity:get_user(user_id=5408af77976f48d5a5cad9ff9a0d20ff)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:19.834543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b12ac-1767-4808-97e7-33a696d19177 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:19.836209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 107/205] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:19 2022] GET /identity/v3/users/5408af77976f48d5a5cad9ff9a0d20ff => generated 289 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:19.841833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320fe71-597b-4509-9367-a062d0d9c399 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:19.971046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d320fe71-597b-4509-9367-a062d0d9c399 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:20.023215 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366e999-f20d-4d2f-a178-1375e520a97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:20.024018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366e999-f20d-4d2f-a178-1375e520a97a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:20.024308 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366e999-f20d-4d2f-a178-1375e520a97a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:20.024582 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6366e999-f20d-4d2f-a178-1375e520a97a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:20.069512 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:20.122019 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366e999-f20d-4d2f-a178-1375e520a97a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:20.123494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6366e999-f20d-4d2f-a178-1375e520a97a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:20.247971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 99/206] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:20.253521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b12ac-1767-4808-97e7-33a696d19177 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:20.289961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:20.362810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b12ac-1767-4808-97e7-33a696d19177 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:20.398537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df64d66-d5fd-4229-bc9f-7b9ca37d6485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:20.399401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df64d66-d5fd-4229-bc9f-7b9ca37d6485 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:20.399687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df64d66-d5fd-4229-bc9f-7b9ca37d6485 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:20.399963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df64d66-d5fd-4229-bc9f-7b9ca37d6485 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:20.401208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df64d66-d5fd-4229-bc9f-7b9ca37d6485 None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:20.402660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df64d66-d5fd-4229-bc9f-7b9ca37d6485 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:20.404698 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 108/207] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:20 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:20.412020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366e999-f20d-4d2f-a178-1375e520a97a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:20.490222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:20.534662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6366e999-f20d-4d2f-a178-1375e520a97a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:20.571784 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2e202-de98-4869-b696-9b2fcaf236e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:20.572449 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2e202-de98-4869-b696-9b2fcaf236e8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:20.572760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2e202-de98-4869-b696-9b2fcaf236e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:20.573041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2e202-de98-4869-b696-9b2fcaf236e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:20.573883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2e202-de98-4869-b696-9b2fcaf236e8 None admin] RBAC: Adding query filter params (role.id=fe57e073b65947b199b35bd49fc6e30a, scope.project.id=ea37fde41a0c43a790da11fb742b5490, user.id=5408af77976f48d5a5cad9ff9a0d20ff) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:20.574195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2e202-de98-4869-b696-9b2fcaf236e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:20.575640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2e202-de98-4869-b696-9b2fcaf236e8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:20.582697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 100/208] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:20 2022] GET /identity/v3/role_assignments?user.id=5408af77976f48d5a5cad9ff9a0d20ff&scope.project.id=ea37fde41a0c43a790da11fb742b5490&role.id=fe57e073b65947b199b35bd49fc6e30a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:20.788476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acf7e7-7950-4fe8-9764-84ab38d8db2c None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:20.788721 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acf7e7-7950-4fe8-9764-84ab38d8db2c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:20.789433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54acf7e7-7950-4fe8-9764-84ab38d8db2c None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:20.790531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 109/209] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:20.797771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cee427-47c0-40c9-8fb7-b99e5bb36e0a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:20.798310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cee427-47c0-40c9-8fb7-b99e5bb36e0a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:20.798826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cee427-47c0-40c9-8fb7-b99e5bb36e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:20.817714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-15cee427-47c0-40c9-8fb7-b99e5bb36e0a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:20.819243 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-15cee427-47c0-40c9-8fb7-b99e5bb36e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:20.888285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 101/210] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:20.915911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54acf7e7-7950-4fe8-9764-84ab38d8db2c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:21.039958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54acf7e7-7950-4fe8-9764-84ab38d8db2c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:21.041935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:21.087295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6e12d8-97c0-41ce-a625-0ff0fb41dfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:21.088106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e12d8-97c0-41ce-a625-0ff0fb41dfa6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:21.088389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e12d8-97c0-41ce-a625-0ff0fb41dfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:21.088662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6e12d8-97c0-41ce-a625-0ff0fb41dfa6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:21.101712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6e12d8-97c0-41ce-a625-0ff0fb41dfa6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:21.112028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9aad8d-7bc6-438d-848b-e36f68e16658 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2b6e12d8-97c0-41ce-a625-0ff0fb41dfa6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:21.112280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:21.113171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:21.113970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:21.113970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:21.113970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:21.113970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:21.113970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:21.113970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:21.113970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:21.113970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:21.113970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:21.113970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:21.113970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:21.113970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:21.114580 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9aad8d-7bc6-438d-848b-e36f68e16658 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:21.114580 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9aad8d-7bc6-438d-848b-e36f68e16658 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:21.114708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 110/211] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:21.114780 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 102/212] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:41:21.120505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9aad8d-7bc6-438d-848b-e36f68e16658 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:21.123237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87f712-306c-483a-94cd-b3054c136ea3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:21.123237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87f712-306c-483a-94cd-b3054c136ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:21.123237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87f712-306c-483a-94cd-b3054c136ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:21.141109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ef87f712-306c-483a-94cd-b3054c136ea3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:21.142626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ef87f712-306c-483a-94cd-b3054c136ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:21.181977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:21.216480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 111/213] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:21.238885 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9aad8d-7bc6-438d-848b-e36f68e16658 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:21.242194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87f712-306c-483a-94cd-b3054c136ea3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:21.281913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1487235-a50a-4588-b3f3-b5af53f044b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:21.282951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1487235-a50a-4588-b3f3-b5af53f044b5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:21.283653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1487235-a50a-4588-b3f3-b5af53f044b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:21.284961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1487235-a50a-4588-b3f3-b5af53f044b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:21.285751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1487235-a50a-4588-b3f3-b5af53f044b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:21.286063 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1487235-a50a-4588-b3f3-b5af53f044b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:21.287451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1487235-a50a-4588-b3f3-b5af53f044b5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:21.304175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 103/214] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:21.310333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1487235-a50a-4588-b3f3-b5af53f044b5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:21.358001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:21.369031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87f712-306c-483a-94cd-b3054c136ea3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:21.407250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69682-ff29-4411-b162-0e3cf5d12b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:21.408167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69682-ff29-4411-b162-0e3cf5d12b1a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:21.408452 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69682-ff29-4411-b162-0e3cf5d12b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:21.408750 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a69682-ff29-4411-b162-0e3cf5d12b1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:21.409936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a69682-ff29-4411-b162-0e3cf5d12b1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:21.411443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a69682-ff29-4411-b162-0e3cf5d12b1a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:21.418120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 112/215] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:21.419622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69682-ff29-4411-b162-0e3cf5d12b1a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:21.425545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1487235-a50a-4588-b3f3-b5af53f044b5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:21.444951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:21.478046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991454ec-9a9f-4e65-8443-80550ae7785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:21.479471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991454ec-9a9f-4e65-8443-80550ae7785b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:21.479830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991454ec-9a9f-4e65-8443-80550ae7785b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:21.480110 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991454ec-9a9f-4e65-8443-80550ae7785b None admin] PATH_INFO: `/v3/users/70d8c50094824daaa570bd5a9b706278` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:21.500991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991454ec-9a9f-4e65-8443-80550ae7785b None admin] RBAC: Authorizing `identity:get_user(user_id=70d8c50094824daaa570bd5a9b706278)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:21.502975 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991454ec-9a9f-4e65-8443-80550ae7785b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:21.505631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 104/216] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:21 2022] GET /identity/v3/users/70d8c50094824daaa570bd5a9b706278 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:21.510964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991454ec-9a9f-4e65-8443-80550ae7785b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:21.545911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a69682-ff29-4411-b162-0e3cf5d12b1a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:21.583232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6e9a4-1d7d-4e86-9896-51e0662bf768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:21.584250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6e9a4-1d7d-4e86-9896-51e0662bf768 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:21.584923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6e9a4-1d7d-4e86-9896-51e0662bf768 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:21.585527 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6e9a4-1d7d-4e86-9896-51e0662bf768 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:21.586435 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6e9a4-1d7d-4e86-9896-51e0662bf768 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:21.588146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b6e9a4-1d7d-4e86-9896-51e0662bf768 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:21.617252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991454ec-9a9f-4e65-8443-80550ae7785b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:21.621044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 113/217] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:41:21 2022] POST /identity/v3/users => generated 297 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:41:21.622495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf3e93-ed8c-433f-a8d4-5a9787eb4413 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:21.622792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf3e93-ed8c-433f-a8d4-5a9787eb4413 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:21.623067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf3e93-ed8c-433f-a8d4-5a9787eb4413 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:21.628782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 114/218] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:21.630318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aecc6-156a-4e7a-88d6-df76f8a2831a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:21.630606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aecc6-156a-4e7a-88d6-df76f8a2831a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:21.630884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56aecc6-156a-4e7a-88d6-df76f8a2831a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:21.652632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d56aecc6-156a-4e7a-88d6-df76f8a2831a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:21.654339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d56aecc6-156a-4e7a-88d6-df76f8a2831a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:21.667922 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cb2d5-decd-4dc1-807b-057edacdfc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:21.668604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cb2d5-decd-4dc1-807b-057edacdfc0b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:21.668920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cb2d5-decd-4dc1-807b-057edacdfc0b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:21.669199 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88cb2d5-decd-4dc1-807b-057edacdfc0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:21.681426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:21.719339 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:21.752159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88cb2d5-decd-4dc1-807b-057edacdfc0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:21.760725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88cb2d5-decd-4dc1-807b-057edacdfc0b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:21.761491 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 115/219] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:21.778437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aecc6-156a-4e7a-88d6-df76f8a2831a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:21.882586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 105/220] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:21.884145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cb2d5-decd-4dc1-807b-057edacdfc0b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:21.899829 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56aecc6-156a-4e7a-88d6-df76f8a2831a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:21.940839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605fe06-4b8f-462e-be64-6d5baae44740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:21.941683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605fe06-4b8f-462e-be64-6d5baae44740 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:21.942027 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605fe06-4b8f-462e-be64-6d5baae44740 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:21.942308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d605fe06-4b8f-462e-be64-6d5baae44740 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:21.955062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d605fe06-4b8f-462e-be64-6d5baae44740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-d605fe06-4b8f-462e-be64-6d5baae44740 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:21.965984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:21.966915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:21.967706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:21.967706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:21.967706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:21.967706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:21.967706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:21.967706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:21.967706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:21.967706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:21.967706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:21.967706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:21.967706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:21.967706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:21.970857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 116/221] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:41:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:41:21.976653 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605fe06-4b8f-462e-be64-6d5baae44740 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:21.977368 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:21.988897 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:22.018982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88cb2d5-decd-4dc1-807b-057edacdfc0b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:22.058417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068192a-bac6-43c0-93a5-045ade637029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:22.059281 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068192a-bac6-43c0-93a5-045ade637029 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:22.059569 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068192a-bac6-43c0-93a5-045ade637029 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:22.059845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e068192a-bac6-43c0-93a5-045ade637029 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:22.061097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068192a-bac6-43c0-93a5-045ade637029 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:22.062626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e068192a-bac6-43c0-93a5-045ade637029 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:22.064700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 106/222] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:21 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:22.071049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068192a-bac6-43c0-93a5-045ade637029 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:22.121638 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d605fe06-4b8f-462e-be64-6d5baae44740 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:22.171379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea3230-e153-476c-a527-4adb2f9d6196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:22.172250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea3230-e153-476c-a527-4adb2f9d6196 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:22.172541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea3230-e153-476c-a527-4adb2f9d6196 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:22.172852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea3230-e153-476c-a527-4adb2f9d6196 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:22.173886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea3230-e153-476c-a527-4adb2f9d6196 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:22.174659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea3230-e153-476c-a527-4adb2f9d6196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:22.176324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea3230-e153-476c-a527-4adb2f9d6196 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:22.187642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 117/223] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:22.193639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea3230-e153-476c-a527-4adb2f9d6196 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:22.199582 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e068192a-bac6-43c0-93a5-045ade637029 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:22.283325 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:22.305420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f4120-5dfc-4157-b74d-a99d4a51f5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:22.306102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f4120-5dfc-4157-b74d-a99d4a51f5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:22.306394 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f4120-5dfc-4157-b74d-a99d4a51f5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:22.306673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f4120-5dfc-4157-b74d-a99d4a51f5c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:22.307524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f4120-5dfc-4157-b74d-a99d4a51f5c8 None admin] RBAC: Adding query filter params (role.id=20703235b4c543bc9c51fb7c26da270d, scope.project.id=fcf4e9bfa6db4a5686e55857c6019793, user.id=70d8c50094824daaa570bd5a9b706278) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:22.307838 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f4120-5dfc-4157-b74d-a99d4a51f5c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:22.309329 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f4120-5dfc-4157-b74d-a99d4a51f5c8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:22.324622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 107/224] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:22 2022] GET /identity/v3/role_assignments?user.id=70d8c50094824daaa570bd5a9b706278&scope.project.id=fcf4e9bfa6db4a5686e55857c6019793&role.id=20703235b4c543bc9c51fb7c26da270d => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:22.327537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea3230-e153-476c-a527-4adb2f9d6196 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:22.366190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99967be-c8d8-4a02-83dc-ff6654b80cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:22.367093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99967be-c8d8-4a02-83dc-ff6654b80cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:22.367385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99967be-c8d8-4a02-83dc-ff6654b80cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:22.367667 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99967be-c8d8-4a02-83dc-ff6654b80cd9 None admin] PATH_INFO: `/v3/users/f859f8bbfa2b425194813beaee73cecf` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:22.368960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99967be-c8d8-4a02-83dc-ff6654b80cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=f859f8bbfa2b425194813beaee73cecf)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:22.370424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99967be-c8d8-4a02-83dc-ff6654b80cd9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:22.372408 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 118/225] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:22 2022] GET /identity/v3/users/f859f8bbfa2b425194813beaee73cecf => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:22.378702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f4120-5dfc-4157-b74d-a99d4a51f5c8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:22.382900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:22.480625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af4b3b-8160-4e8c-9a19-634077dc20c8 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:22.480985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af4b3b-8160-4e8c-9a19-634077dc20c8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:22.481243 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af4b3b-8160-4e8c-9a19-634077dc20c8 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:22.489802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 119/226] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:22.495328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f4120-5dfc-4157-b74d-a99d4a51f5c8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:22.502917 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f66f13-7579-4a60-ad04-32cdeaa19034 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:22.503221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f66f13-7579-4a60-ad04-32cdeaa19034 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:22.503496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f66f13-7579-4a60-ad04-32cdeaa19034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:22.540910 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-30f66f13-7579-4a60-ad04-32cdeaa19034 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:22.542671 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-30f66f13-7579-4a60-ad04-32cdeaa19034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:22.560278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d44751-5a52-49be-9001-e16e8430a791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:22.561183 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d44751-5a52-49be-9001-e16e8430a791 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:22.561471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d44751-5a52-49be-9001-e16e8430a791 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:22.561752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d44751-5a52-49be-9001-e16e8430a791 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:22.564295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d44751-5a52-49be-9001-e16e8430a791 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:22.565798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d44751-5a52-49be-9001-e16e8430a791 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:22.567695 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 108/227] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:22 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:22.574524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d44751-5a52-49be-9001-e16e8430a791 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:22.643117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 120/228] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:22.695903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f66f13-7579-4a60-ad04-32cdeaa19034 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:22.700716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:22.738260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d44751-5a52-49be-9001-e16e8430a791 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:22.797564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:22.809009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee123d50-62e4-4638-84b8-0dd38f28a633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:22.814977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee123d50-62e4-4638-84b8-0dd38f28a633 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:22.815501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee123d50-62e4-4638-84b8-0dd38f28a633 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:22.815974 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee123d50-62e4-4638-84b8-0dd38f28a633 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793/users/f859f8bbfa2b425194813beaee73cecf/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:22.818472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee123d50-62e4-4638-84b8-0dd38f28a633 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf4e9bfa6db4a5686e55857c6019793, user_id=f859f8bbfa2b425194813beaee73cecf, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:22.820517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee123d50-62e4-4638-84b8-0dd38f28a633 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:22.844701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 109/229] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:41:22 2022] PUT /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793/users/f859f8bbfa2b425194813beaee73cecf/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:41:22.871134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f66f13-7579-4a60-ad04-32cdeaa19034 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:22.921181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51f956-c09f-4db3-ab21-69c730356a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:22.922976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51f956-c09f-4db3-ab21-69c730356a0f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:22.923485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51f956-c09f-4db3-ab21-69c730356a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:22.923978 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51f956-c09f-4db3-ab21-69c730356a0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:22.941760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51f956-c09f-4db3-ab21-69c730356a0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8f51f956-c09f-4db3-ab21-69c730356a0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:22.950470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:22.951342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:22.952123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:22.952123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:22.952123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:22.952123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:22.952123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:22.952123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:22.952123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:22.952123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:22.952123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:22.952123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:22.952123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:22.952123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:22.954593 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 121/230] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:22.960658 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee123d50-62e4-4638-84b8-0dd38f28a633 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:23.093665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee123d50-62e4-4638-84b8-0dd38f28a633 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:23.129574 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:23.137761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fe1c0-8b1f-41ed-b2fc-c44ec26be4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:23.138722 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fe1c0-8b1f-41ed-b2fc-c44ec26be4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:23.139200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fe1c0-8b1f-41ed-b2fc-c44ec26be4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:23.139770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fe1c0-8b1f-41ed-b2fc-c44ec26be4b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:23.140818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fe1c0-8b1f-41ed-b2fc-c44ec26be4b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:23.141328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fe1c0-8b1f-41ed-b2fc-c44ec26be4b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:23.142915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fe1c0-8b1f-41ed-b2fc-c44ec26be4b4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:23.152451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 110/231] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:23.159321 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51f956-c09f-4db3-ab21-69c730356a0f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:23.299825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51f956-c09f-4db3-ab21-69c730356a0f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:23.335706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:23.346891 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccabe086-65a9-4cb6-b2f9-185d9c02ad43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:23.348046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabe086-65a9-4cb6-b2f9-185d9c02ad43 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:23.348581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabe086-65a9-4cb6-b2f9-185d9c02ad43 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:23.349133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccabe086-65a9-4cb6-b2f9-185d9c02ad43 None admin] PATH_INFO: `/v3/users/f859f8bbfa2b425194813beaee73cecf` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:23.350584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccabe086-65a9-4cb6-b2f9-185d9c02ad43 None admin] RBAC: Authorizing `identity:get_user(user_id=f859f8bbfa2b425194813beaee73cecf)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:23.352297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccabe086-65a9-4cb6-b2f9-185d9c02ad43 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:23.354887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 122/232] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:23 2022] GET /identity/v3/users/f859f8bbfa2b425194813beaee73cecf => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:23.361133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fe1c0-8b1f-41ed-b2fc-c44ec26be4b4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:23.466661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fe1c0-8b1f-41ed-b2fc-c44ec26be4b4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:23.502679 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3eed39-d124-45bc-b209-b2b33fd66b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:23.503961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3eed39-d124-45bc-b209-b2b33fd66b64 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:23.504471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3eed39-d124-45bc-b209-b2b33fd66b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:23.504984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3eed39-d124-45bc-b209-b2b33fd66b64 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:23.506375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3eed39-d124-45bc-b209-b2b33fd66b64 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:23.508013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3eed39-d124-45bc-b209-b2b33fd66b64 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:23.510383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 111/233] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:23 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:23.517009 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccabe086-65a9-4cb6-b2f9-185d9c02ad43 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:23.648819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccabe086-65a9-4cb6-b2f9-185d9c02ad43 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:23.706603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d2391-e972-40de-8c0f-0affaa4cb459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:23.707888 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d2391-e972-40de-8c0f-0affaa4cb459 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:23.708389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d2391-e972-40de-8c0f-0affaa4cb459 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:23.709069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d2391-e972-40de-8c0f-0affaa4cb459 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793/users/f859f8bbfa2b425194813beaee73cecf/roles/2d0e0231af5d4e51b80de6bb65e8b02b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:23.713954 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d2391-e972-40de-8c0f-0affaa4cb459 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf4e9bfa6db4a5686e55857c6019793, user_id=f859f8bbfa2b425194813beaee73cecf, role_id=2d0e0231af5d4e51b80de6bb65e8b02b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:23.715614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d2391-e972-40de-8c0f-0affaa4cb459 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:23.741913 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 123/234] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:41:23 2022] PUT /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793/users/f859f8bbfa2b425194813beaee73cecf/roles/2d0e0231af5d4e51b80de6bb65e8b02b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:41:24.326197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516a059-5790-4b1d-b799-9ba487ea9e91 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:24.326732 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516a059-5790-4b1d-b799-9ba487ea9e91 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:24.326732 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516a059-5790-4b1d-b799-9ba487ea9e91 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:24.328400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 112/235] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:24.334553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f529ba-e85d-40fa-b8f3-7bdee3cfd474 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:24.334847 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f529ba-e85d-40fa-b8f3-7bdee3cfd474 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:24.335134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f529ba-e85d-40fa-b8f3-7bdee3cfd474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:24.361819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d4f529ba-e85d-40fa-b8f3-7bdee3cfd474 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:24.363101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d4f529ba-e85d-40fa-b8f3-7bdee3cfd474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:24.423411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:24.446085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 124/236] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:24.483520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516a059-5790-4b1d-b799-9ba487ea9e91 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:24.552837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302c4cd-8ab2-4f7d-874f-e93b2f9b0734 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:24.553346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302c4cd-8ab2-4f7d-874f-e93b2f9b0734 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:24.553420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302c4cd-8ab2-4f7d-874f-e93b2f9b0734 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:24.554796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 125/237] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:24.561456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6b09b-3fc1-476b-85b9-f63d0628b4d7 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:24.563684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6b09b-3fc1-476b-85b9-f63d0628b4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:24.564162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6b09b-3fc1-476b-85b9-f63d0628b4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:24.591577 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:24.599459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5ac6b09b-3fc1-476b-85b9-f63d0628b4d7 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:24.601093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5ac6b09b-3fc1-476b-85b9-f63d0628b4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:24.684306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 126/238] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:24.690486 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516a059-5790-4b1d-b799-9ba487ea9e91 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:24.708909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6b09b-3fc1-476b-85b9-f63d0628b4d7 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:24.756554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d68e7f-2485-40c1-8298-3ac9bfdb4b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:24.757617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d68e7f-2485-40c1-8298-3ac9bfdb4b08 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:24.758081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d68e7f-2485-40c1-8298-3ac9bfdb4b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:24.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d68e7f-2485-40c1-8298-3ac9bfdb4b08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:24.759843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d68e7f-2485-40c1-8298-3ac9bfdb4b08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:24.761477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d68e7f-2485-40c1-8298-3ac9bfdb4b08 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:24.763276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 113/239] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:24.770852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d68e7f-2485-40c1-8298-3ac9bfdb4b08 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:24.827191 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6b09b-3fc1-476b-85b9-f63d0628b4d7 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:24.842546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:24.876466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8903d6-c191-4b59-8b70-84917dcdc297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:24.877796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8903d6-c191-4b59-8b70-84917dcdc297 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:24.878307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8903d6-c191-4b59-8b70-84917dcdc297 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:24.878776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8903d6-c191-4b59-8b70-84917dcdc297 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:24.901175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8903d6-c191-4b59-8b70-84917dcdc297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-1a8903d6-c191-4b59-8b70-84917dcdc297 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:24.910295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:24.911183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:24.911969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 127/240] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:24.913345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff8b1-f0c9-4ac5-88be-abe0f90ac539 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:24.913345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff8b1-f0c9-4ac5-88be-abe0f90ac539 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:24.913345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ff8b1-f0c9-4ac5-88be-abe0f90ac539 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:24.920438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 128/241] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:24.920438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1e5d6-52e8-48a2-bfe7-4bce8901aa01 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:24.920792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1e5d6-52e8-48a2-bfe7-4bce8901aa01 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:24.921007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1e5d6-52e8-48a2-bfe7-4bce8901aa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:24.942963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-72d1e5d6-52e8-48a2-bfe7-4bce8901aa01 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:24.943666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d68e7f-2485-40c1-8298-3ac9bfdb4b08 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:24.944164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-72d1e5d6-52e8-48a2-bfe7-4bce8901aa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:24.975903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:24.983970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d354e3-5c12-43e5-ae3f-822dc2039abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:24.985111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d354e3-5c12-43e5-ae3f-822dc2039abf None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:24.985595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d354e3-5c12-43e5-ae3f-822dc2039abf None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:24.986075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d354e3-5c12-43e5-ae3f-822dc2039abf None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:24.986890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d354e3-5c12-43e5-ae3f-822dc2039abf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:24.988604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d354e3-5c12-43e5-ae3f-822dc2039abf None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:25.011942 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 114/242] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:41:24 2022] POST /identity/v3/users => generated 293 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:41:25.017338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d354e3-5c12-43e5-ae3f-822dc2039abf None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:25.029030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 129/243] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:25.054085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1e5d6-52e8-48a2-bfe7-4bce8901aa01 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:25.180004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d354e3-5c12-43e5-ae3f-822dc2039abf None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:25.196702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1e5d6-52e8-48a2-bfe7-4bce8901aa01 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:25.200369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:25.222627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81b7d3-b437-4dfb-9634-7f34063e71d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:25.223716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81b7d3-b437-4dfb-9634-7f34063e71d6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:25.224285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81b7d3-b437-4dfb-9634-7f34063e71d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:25.224832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81b7d3-b437-4dfb-9634-7f34063e71d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:25.226016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81b7d3-b437-4dfb-9634-7f34063e71d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:25.226556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81b7d3-b437-4dfb-9634-7f34063e71d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:25.228279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81b7d3-b437-4dfb-9634-7f34063e71d6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:25.239544 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 115/244] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:25.245302 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81b7d3-b437-4dfb-9634-7f34063e71d6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:25.262769 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e389f-8574-4497-b234-a87d27807aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:25.263583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e389f-8574-4497-b234-a87d27807aba None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:25.263875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e389f-8574-4497-b234-a87d27807aba None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:25.264153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e389f-8574-4497-b234-a87d27807aba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:25.278582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e389f-8574-4497-b234-a87d27807aba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-197e389f-8574-4497-b234-a87d27807aba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:25.286820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:25.287903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:25.288983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 130/245] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:41:25.297461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e389f-8574-4497-b234-a87d27807aba None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:25.303936 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:25.383129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81b7d3-b437-4dfb-9634-7f34063e71d6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:25.423691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e2d0e-8a4a-43ea-93cf-bf783bd57245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:25.424797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e2d0e-8a4a-43ea-93cf-bf783bd57245 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:25.425273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e2d0e-8a4a-43ea-93cf-bf783bd57245 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:25.425675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e2d0e-8a4a-43ea-93cf-bf783bd57245 None admin] PATH_INFO: `/v3/users/90db243021dc458bb98bd7d1760e2613` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:25.433248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e389f-8574-4497-b234-a87d27807aba None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:25.446931 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e2d0e-8a4a-43ea-93cf-bf783bd57245 None admin] RBAC: Authorizing `identity:get_user(user_id=90db243021dc458bb98bd7d1760e2613)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:25.448971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e2d0e-8a4a-43ea-93cf-bf783bd57245 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:25.451274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 116/246] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:25 2022] GET /identity/v3/users/90db243021dc458bb98bd7d1760e2613 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:25.458017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e2d0e-8a4a-43ea-93cf-bf783bd57245 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:25.478265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f62bb6-c6f6-4022-86f7-a612998ea986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:25.478949 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f62bb6-c6f6-4022-86f7-a612998ea986 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:25.479240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f62bb6-c6f6-4022-86f7-a612998ea986 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:25.479523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f62bb6-c6f6-4022-86f7-a612998ea986 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:25.480902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f62bb6-c6f6-4022-86f7-a612998ea986 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:25.481221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f62bb6-c6f6-4022-86f7-a612998ea986 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:25.482811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f62bb6-c6f6-4022-86f7-a612998ea986 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:25.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 131/247] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:25.500729 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f62bb6-c6f6-4022-86f7-a612998ea986 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:25.534344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:25.570147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e2d0e-8a4a-43ea-93cf-bf783bd57245 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:25.579782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:25.617733 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278f5a4b-c268-4fb2-83d1-4d0bc0651b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:25.618444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f5a4b-c268-4fb2-83d1-4d0bc0651b2d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:25.618730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f5a4b-c268-4fb2-83d1-4d0bc0651b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:25.619004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278f5a4b-c268-4fb2-83d1-4d0bc0651b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:25.643294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f62bb6-c6f6-4022-86f7-a612998ea986 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:25.691672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9eee12-3513-4dff-a49e-388ea6a0df78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:25.692921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9eee12-3513-4dff-a49e-388ea6a0df78 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:25.693380 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9eee12-3513-4dff-a49e-388ea6a0df78 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:25.693791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9eee12-3513-4dff-a49e-388ea6a0df78 None admin] PATH_INFO: `/v3/users/70d8c50094824daaa570bd5a9b706278` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:25.696236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9eee12-3513-4dff-a49e-388ea6a0df78 None admin] RBAC: Authorizing `identity:get_user(user_id=70d8c50094824daaa570bd5a9b706278)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:25.699263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9eee12-3513-4dff-a49e-388ea6a0df78 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:25.703508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 132/248] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:25 2022] GET /identity/v3/users/70d8c50094824daaa570bd5a9b706278 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:25.706147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51de60-e567-465b-aa50-0059f007ea2e None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:25.707093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51de60-e567-465b-aa50-0059f007ea2e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:25.707801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51de60-e567-465b-aa50-0059f007ea2e None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:25.709515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f5a4b-c268-4fb2-83d1-4d0bc0651b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:25.710622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 133/249] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:25.711472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278f5a4b-c268-4fb2-83d1-4d0bc0651b2d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:25.711953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51de60-e567-465b-aa50-0059f007ea2e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:25.821817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:25.831261 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51de60-e567-465b-aa50-0059f007ea2e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:25.842538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 117/250] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 11:41:25.843692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9ec48-d63f-4622-b165-998163cf3723 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:25.843985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9ec48-d63f-4622-b165-998163cf3723 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:25.844575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9ec48-d63f-4622-b165-998163cf3723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:25.872351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:25.886717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e2c9ec48-d63f-4622-b165-998163cf3723 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:25.887933 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e2c9ec48-d63f-4622-b165-998163cf3723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:25.902435 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21a4d5-db0b-47c2-82b5-4118cc48da01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:25.903306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21a4d5-db0b-47c2-82b5-4118cc48da01 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:25.903587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21a4d5-db0b-47c2-82b5-4118cc48da01 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:25.903879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21a4d5-db0b-47c2-82b5-4118cc48da01 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:25.905087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21a4d5-db0b-47c2-82b5-4118cc48da01 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:25.909182 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21a4d5-db0b-47c2-82b5-4118cc48da01 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:25.910953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 134/251] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:25 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:25.911734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21a4d5-db0b-47c2-82b5-4118cc48da01 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:26.019720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 118/252] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:26.020985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9ec48-d63f-4622-b165-998163cf3723 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:26.097941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21a4d5-db0b-47c2-82b5-4118cc48da01 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:26.132826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9ec48-d63f-4622-b165-998163cf3723 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:26.162250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6876d-f563-4344-8a4d-5ca504e97d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:26.163174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6876d-f563-4344-8a4d-5ca504e97d34 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:26.163395 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6876d-f563-4344-8a4d-5ca504e97d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:26.163672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6876d-f563-4344-8a4d-5ca504e97d34 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:26.164920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6876d-f563-4344-8a4d-5ca504e97d34 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:26.166378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba6876d-f563-4344-8a4d-5ca504e97d34 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:26.168107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 135/253] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:25 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:26.169248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6876d-f563-4344-8a4d-5ca504e97d34 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:26.180527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5b41a-410a-43b7-a3c5-242023616e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:26.181653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5b41a-410a-43b7-a3c5-242023616e41 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:26.182042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5b41a-410a-43b7-a3c5-242023616e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:26.182425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5b41a-410a-43b7-a3c5-242023616e41 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793/users/70d8c50094824daaa570bd5a9b706278/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:26.188806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5b41a-410a-43b7-a3c5-242023616e41 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf4e9bfa6db4a5686e55857c6019793, user_id=70d8c50094824daaa570bd5a9b706278, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:26.190429 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5b41a-410a-43b7-a3c5-242023616e41 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:26.201193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:26.210460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:26.217986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 119/254] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:41:26 2022] PUT /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793/users/70d8c50094824daaa570bd5a9b706278/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:41:26.219497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5b41a-410a-43b7-a3c5-242023616e41 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:26.370303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5b41a-410a-43b7-a3c5-242023616e41 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:26.398723 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6876d-f563-4344-8a4d-5ca504e97d34 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:26.412927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10967b8-6f4a-4b05-a6a9-a163c6929977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:26.413609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10967b8-6f4a-4b05-a6a9-a163c6929977 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:26.413878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10967b8-6f4a-4b05-a6a9-a163c6929977 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:26.414156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10967b8-6f4a-4b05-a6a9-a163c6929977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:26.415010 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10967b8-6f4a-4b05-a6a9-a163c6929977 None admin] RBAC: Adding query filter params (role.id=fe57e073b65947b199b35bd49fc6e30a, scope.project.id=fcf4e9bfa6db4a5686e55857c6019793, user.id=90db243021dc458bb98bd7d1760e2613) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:26.415325 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10967b8-6f4a-4b05-a6a9-a163c6929977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:26.416857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10967b8-6f4a-4b05-a6a9-a163c6929977 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:26.428834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 120/255] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:26 2022] GET /identity/v3/role_assignments?user.id=90db243021dc458bb98bd7d1760e2613&scope.project.id=fcf4e9bfa6db4a5686e55857c6019793&role.id=fe57e073b65947b199b35bd49fc6e30a => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:26.430435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e832b1-344b-4676-ac35-163b1c3603f4 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:26.430756 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e832b1-344b-4676-ac35-163b1c3603f4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:26.431042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e832b1-344b-4676-ac35-163b1c3603f4 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:26.432942 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 121/256] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:26.450546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b89ef6-dc9f-4b0a-8731-ce809131c69e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:26.450857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b89ef6-dc9f-4b0a-8731-ce809131c69e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:26.451120 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b89ef6-dc9f-4b0a-8731-ce809131c69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:26.483120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5315c5-f152-4b29-b7f8-fa06a01cdb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:26.484510 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5315c5-f152-4b29-b7f8-fa06a01cdb5b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:26.484826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5315c5-f152-4b29-b7f8-fa06a01cdb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:26.485108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5315c5-f152-4b29-b7f8-fa06a01cdb5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:26.485689 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a7b89ef6-dc9f-4b0a-8731-ce809131c69e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:26.486905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a7b89ef6-dc9f-4b0a-8731-ce809131c69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:26.505587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5315c5-f152-4b29-b7f8-fa06a01cdb5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-5a5315c5-f152-4b29-b7f8-fa06a01cdb5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:26.514090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:26.515067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:26.515881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:26.515881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:26.515881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:26.515881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:26.515881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:26.515881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:26.515881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:26.515881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:26.515881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:26.515881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:26.515881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:26.515881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:26.517965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 136/257] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:41:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:26.524402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5315c5-f152-4b29-b7f8-fa06a01cdb5b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:26.600350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:26.625579 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 122/258] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:26.671699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b89ef6-dc9f-4b0a-8731-ce809131c69e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:26.717043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:26.735116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5315c5-f152-4b29-b7f8-fa06a01cdb5b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:26.786292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f415f-32af-4168-9618-dcbaf2fb6309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:26.787198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f415f-32af-4168-9618-dcbaf2fb6309 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:26.787484 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f415f-32af-4168-9618-dcbaf2fb6309 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:26.787759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f415f-32af-4168-9618-dcbaf2fb6309 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:26.788522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f415f-32af-4168-9618-dcbaf2fb6309 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:26.788857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f415f-32af-4168-9618-dcbaf2fb6309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:26.790368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f415f-32af-4168-9618-dcbaf2fb6309 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:26.803934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 137/259] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:26.809307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f415f-32af-4168-9618-dcbaf2fb6309 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:26.874851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b89ef6-dc9f-4b0a-8731-ce809131c69e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:26.916912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f415f-32af-4168-9618-dcbaf2fb6309 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:26.952617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ccc7c-72ce-44c1-9725-a5a966379097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:26.953525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ccc7c-72ce-44c1-9725-a5a966379097 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:26.953814 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ccc7c-72ce-44c1-9725-a5a966379097 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:26.954096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ccc7c-72ce-44c1-9725-a5a966379097 None admin] PATH_INFO: `/v3/users/f859f8bbfa2b425194813beaee73cecf` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:26.955309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ccc7c-72ce-44c1-9725-a5a966379097 None admin] RBAC: Authorizing `identity:get_user(user_id=f859f8bbfa2b425194813beaee73cecf)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:26.957883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ccc7c-72ce-44c1-9725-a5a966379097 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:26.959594 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 138/260] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:26 2022] GET /identity/v3/users/f859f8bbfa2b425194813beaee73cecf => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:26.964927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ccc7c-72ce-44c1-9725-a5a966379097 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:26.989070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:26.993960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794c880-8c4a-481d-a1a0-340020619c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:26.994840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794c880-8c4a-481d-a1a0-340020619c79 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:26.995084 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794c880-8c4a-481d-a1a0-340020619c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:26.995368 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3794c880-8c4a-481d-a1a0-340020619c79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:27.033333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3794c880-8c4a-481d-a1a0-340020619c79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-3794c880-8c4a-481d-a1a0-340020619c79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:27.041632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:27.042624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:27.043421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 123/261] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:41:27.054899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794c880-8c4a-481d-a1a0-340020619c79 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:27.077189 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ccc7c-72ce-44c1-9725-a5a966379097 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:27.115096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165bf9c-47f7-49a3-82b2-347a956d6c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:27.115760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165bf9c-47f7-49a3-82b2-347a956d6c41 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:27.116101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165bf9c-47f7-49a3-82b2-347a956d6c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:27.116484 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d165bf9c-47f7-49a3-82b2-347a956d6c41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:27.144138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:27.188659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d165bf9c-47f7-49a3-82b2-347a956d6c41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:27.190081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d165bf9c-47f7-49a3-82b2-347a956d6c41 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:27.213024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:27.222078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3794c880-8c4a-481d-a1a0-340020619c79 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:27.292070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab70cdb-5ffd-415f-a39c-50209957df08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:27.293284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab70cdb-5ffd-415f-a39c-50209957df08 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:27.293779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab70cdb-5ffd-415f-a39c-50209957df08 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:27.294279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab70cdb-5ffd-415f-a39c-50209957df08 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:27.295247 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab70cdb-5ffd-415f-a39c-50209957df08 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:27.295762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab70cdb-5ffd-415f-a39c-50209957df08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:27.296349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 139/262] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:27.297684 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab70cdb-5ffd-415f-a39c-50209957df08 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:27.301760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165bf9c-47f7-49a3-82b2-347a956d6c41 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:27.313918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 124/263] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:27.333790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab70cdb-5ffd-415f-a39c-50209957df08 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:27.413879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d165bf9c-47f7-49a3-82b2-347a956d6c41 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:27.457152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:27.478657 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dd2e3-8778-44ae-a197-4ec8ee37a1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:27.479529 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dd2e3-8778-44ae-a197-4ec8ee37a1ee None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:27.479809 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dd2e3-8778-44ae-a197-4ec8ee37a1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:27.480083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dd2e3-8778-44ae-a197-4ec8ee37a1ee None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:27.482514 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dd2e3-8778-44ae-a197-4ec8ee37a1ee None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:27.484000 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dd2e3-8778-44ae-a197-4ec8ee37a1ee None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:27.491199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 140/264] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:27 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:27.499855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dd2e3-8778-44ae-a197-4ec8ee37a1ee None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:27.547190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab70cdb-5ffd-415f-a39c-50209957df08 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:27.574409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:27.614346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec688-b182-410a-8216-8f91edd39cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:27.615229 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec688-b182-410a-8216-8f91edd39cf0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:27.615507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec688-b182-410a-8216-8f91edd39cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:27.615777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3ec688-b182-410a-8216-8f91edd39cf0 None admin] PATH_INFO: `/v3/users/f859f8bbfa2b425194813beaee73cecf` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:27.618672 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ec688-b182-410a-8216-8f91edd39cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=f859f8bbfa2b425194813beaee73cecf)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:27.620141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3ec688-b182-410a-8216-8f91edd39cf0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:27.621856 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 125/265] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:27 2022] GET /identity/v3/users/f859f8bbfa2b425194813beaee73cecf => generated 283 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:27.645726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec688-b182-410a-8216-8f91edd39cf0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:27.690188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dd2e3-8778-44ae-a197-4ec8ee37a1ee None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:27.756912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badcb7ee-c78a-4005-84ad-147a6d1b8e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:27.757567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badcb7ee-c78a-4005-84ad-147a6d1b8e0e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:27.757846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badcb7ee-c78a-4005-84ad-147a6d1b8e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:27.758124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badcb7ee-c78a-4005-84ad-147a6d1b8e0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:27.758960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badcb7ee-c78a-4005-84ad-147a6d1b8e0e None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=fcf4e9bfa6db4a5686e55857c6019793, user.id=f859f8bbfa2b425194813beaee73cecf) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:27.759272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badcb7ee-c78a-4005-84ad-147a6d1b8e0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:27.762759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badcb7ee-c78a-4005-84ad-147a6d1b8e0e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:27.774379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 141/266] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:27 2022] GET /identity/v3/role_assignments?user.id=f859f8bbfa2b425194813beaee73cecf&scope.project.id=fcf4e9bfa6db4a5686e55857c6019793&role.id=569707378794437b93d168fc8bb9cbc3 => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:27.833024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3ec688-b182-410a-8216-8f91edd39cf0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:27.886404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298704d-2ddf-4744-9cf9-63a9d500d630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:27.887068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298704d-2ddf-4744-9cf9-63a9d500d630 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:27.887399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298704d-2ddf-4744-9cf9-63a9d500d630 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:27.887681 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298704d-2ddf-4744-9cf9-63a9d500d630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:27.954052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:27.982976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298704d-2ddf-4744-9cf9-63a9d500d630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:27.985343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298704d-2ddf-4744-9cf9-63a9d500d630 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:27.994730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963c109-72e9-4322-98d3-7b566d204787 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:27.995045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963c109-72e9-4322-98d3-7b566d204787 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:27.995293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c963c109-72e9-4322-98d3-7b566d204787 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:27.997088 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 142/267] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:28.004505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6ad35-cba8-4e6e-acc8-59e4a0dd3b05 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:28.004847 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6ad35-cba8-4e6e-acc8-59e4a0dd3b05 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:28.005106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6ad35-cba8-4e6e-acc8-59e4a0dd3b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:28.041769 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a7a6ad35-cba8-4e6e-acc8-59e4a0dd3b05 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:28.042907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a7a6ad35-cba8-4e6e-acc8-59e4a0dd3b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:28.137864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:28.159163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 126/268] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 11:41:28.164959 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298704d-2ddf-4744-9cf9-63a9d500d630 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:28.172600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 143/269] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:28.206445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6ad35-cba8-4e6e-acc8-59e4a0dd3b05 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:28.278796 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:28.303839 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298704d-2ddf-4744-9cf9-63a9d500d630 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:28.351726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd007306-2b1d-4d2d-b8a6-8cbe279da2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:28.352632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd007306-2b1d-4d2d-b8a6-8cbe279da2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:28.353156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd007306-2b1d-4d2d-b8a6-8cbe279da2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:28.353448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd007306-2b1d-4d2d-b8a6-8cbe279da2a9 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:28.355467 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd007306-2b1d-4d2d-b8a6-8cbe279da2a9 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:28.360094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd007306-2b1d-4d2d-b8a6-8cbe279da2a9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:28.363978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 127/270] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:28 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:28.370136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd007306-2b1d-4d2d-b8a6-8cbe279da2a9 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:28.411939 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6ad35-cba8-4e6e-acc8-59e4a0dd3b05 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:28.477389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fa7b7-5132-4ea5-ba7c-c2c8f1323947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:28.478306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fa7b7-5132-4ea5-ba7c-c2c8f1323947 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:28.478598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fa7b7-5132-4ea5-ba7c-c2c8f1323947 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:28.478878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fa7b7-5132-4ea5-ba7c-c2c8f1323947 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:28.481997 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fa7b7-5132-4ea5-ba7c-c2c8f1323947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:28.483472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fa7b7-5132-4ea5-ba7c-c2c8f1323947 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:28.486123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 144/271] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:28.491871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd007306-2b1d-4d2d-b8a6-8cbe279da2a9 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:28.492407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fa7b7-5132-4ea5-ba7c-c2c8f1323947 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:28.538529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce338ae-dd31-4766-b364-d377e3df8f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:28.539195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce338ae-dd31-4766-b364-d377e3df8f80 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:28.539651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce338ae-dd31-4766-b364-d377e3df8f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:28.539933 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce338ae-dd31-4766-b364-d377e3df8f80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:28.541380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce338ae-dd31-4766-b364-d377e3df8f80 None admin] RBAC: Adding query filter params (role.id=2d0e0231af5d4e51b80de6bb65e8b02b, scope.project.id=fcf4e9bfa6db4a5686e55857c6019793, user.id=f859f8bbfa2b425194813beaee73cecf) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:28.541723 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce338ae-dd31-4766-b364-d377e3df8f80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:28.543176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce338ae-dd31-4766-b364-d377e3df8f80 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:28.551627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 128/272] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:28 2022] GET /identity/v3/role_assignments?user.id=f859f8bbfa2b425194813beaee73cecf&scope.project.id=fcf4e9bfa6db4a5686e55857c6019793&role.id=2d0e0231af5d4e51b80de6bb65e8b02b => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:28.553502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e4292-f6d9-467d-9b24-8b1d4e968deb None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:28.553803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e4292-f6d9-467d-9b24-8b1d4e968deb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:28.554145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e4292-f6d9-467d-9b24-8b1d4e968deb None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:28.555517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 129/273] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:28.570427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:28.573465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a5177-3216-4a62-956b-eacad7f8b542 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:28.573759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a5177-3216-4a62-956b-eacad7f8b542 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:28.574192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a5177-3216-4a62-956b-eacad7f8b542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:28.601954 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-345a5177-3216-4a62-956b-eacad7f8b542 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:28.603226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-345a5177-3216-4a62-956b-eacad7f8b542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:28.620089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:28.647555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fa7b7-5132-4ea5-ba7c-c2c8f1323947 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:28.714969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2b768-c27d-4a4f-b123-1ef473e1568d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:28.715660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2b768-c27d-4a4f-b123-1ef473e1568d None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:28.717354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2b768-c27d-4a4f-b123-1ef473e1568d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:28.717354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2b768-c27d-4a4f-b123-1ef473e1568d None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:28.717354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2b768-c27d-4a4f-b123-1ef473e1568d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:28.719834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2b768-c27d-4a4f-b123-1ef473e1568d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:28.747225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 145/274] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:41:28 2022] POST /identity/v3/users => generated 293 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:41:28.776299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 130/275] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:28.803674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2b768-c27d-4a4f-b123-1ef473e1568d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:28.934429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2b768-c27d-4a4f-b123-1ef473e1568d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:28.943220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:28.976557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6846553-c50c-49ed-be86-6da1f1d32c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:28.977402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6846553-c50c-49ed-be86-6da1f1d32c3b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:28.977688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6846553-c50c-49ed-be86-6da1f1d32c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:28.978037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6846553-c50c-49ed-be86-6da1f1d32c3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:28.989802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6846553-c50c-49ed-be86-6da1f1d32c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e6846553-c50c-49ed-be86-6da1f1d32c3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:28.997850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:28.998779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:28.999679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 146/276] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:29.006541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a5177-3216-4a62-956b-eacad7f8b542 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:29.120553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a5177-3216-4a62-956b-eacad7f8b542 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:29.159482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7041b27d-3d50-4b32-9e39-91a3caab1d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:29.160163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041b27d-3d50-4b32-9e39-91a3caab1d88 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:29.160449 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041b27d-3d50-4b32-9e39-91a3caab1d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:29.162327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7041b27d-3d50-4b32-9e39-91a3caab1d88 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:29.162327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041b27d-3d50-4b32-9e39-91a3caab1d88 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:29.162327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041b27d-3d50-4b32-9e39-91a3caab1d88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:29.163678 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7041b27d-3d50-4b32-9e39-91a3caab1d88 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:29.172970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 131/277] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:29.179014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6846553-c50c-49ed-be86-6da1f1d32c3b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:29.298815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6846553-c50c-49ed-be86-6da1f1d32c3b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:29.335621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690adbd0-3f63-4980-b366-d044fcddd140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:29.336810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690adbd0-3f63-4980-b366-d044fcddd140 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:29.337252 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690adbd0-3f63-4980-b366-d044fcddd140 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:29.337694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690adbd0-3f63-4980-b366-d044fcddd140 None admin] PATH_INFO: `/v3/users/72ce33c269464062a58d62a2246da7a1` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:29.348456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805df73-1730-4983-96ff-ec3edbf6520d None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:29.348907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805df73-1730-4983-96ff-ec3edbf6520d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:29.349290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805df73-1730-4983-96ff-ec3edbf6520d None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:29.351018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 132/278] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:29.356736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e659fbe-70fd-4fde-89a5-f04999658cbd None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:29.357136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e659fbe-70fd-4fde-89a5-f04999658cbd None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:29.357519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e659fbe-70fd-4fde-89a5-f04999658cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:29.362474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690adbd0-3f63-4980-b366-d044fcddd140 None admin] RBAC: Authorizing `identity:get_user(user_id=72ce33c269464062a58d62a2246da7a1)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:29.364226 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690adbd0-3f63-4980-b366-d044fcddd140 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:29.370215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 147/279] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:29 2022] GET /identity/v3/users/72ce33c269464062a58d62a2246da7a1 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:29.372634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690adbd0-3f63-4980-b366-d044fcddd140 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:29.373293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:29.406037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7e659fbe-70fd-4fde-89a5-f04999658cbd None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:29.406991 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:29.408080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7e659fbe-70fd-4fde-89a5-f04999658cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:29.520260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690adbd0-3f63-4980-b366-d044fcddd140 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:29.528785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 133/280] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:29.569711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edafda-b7a1-490a-8f35-92fba89f4f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:29.570617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edafda-b7a1-490a-8f35-92fba89f4f1c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:29.571041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edafda-b7a1-490a-8f35-92fba89f4f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:29.571455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edafda-b7a1-490a-8f35-92fba89f4f1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:29.572433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edafda-b7a1-490a-8f35-92fba89f4f1c None admin] RBAC: Adding query filter params (role.id=20703235b4c543bc9c51fb7c26da270d, user.id=72ce33c269464062a58d62a2246da7a1, scope.system=all) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:29.572913 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edafda-b7a1-490a-8f35-92fba89f4f1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:29.574575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edafda-b7a1-490a-8f35-92fba89f4f1c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:29.583236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 148/281] 10.222.0.67 () {56 vars in 1308 bytes} [Thu Jun 23 11:41:29 2022] GET /identity/v3/role_assignments?user.id=72ce33c269464062a58d62a2246da7a1&scope.system=all&role.id=20703235b4c543bc9c51fb7c26da270d => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:29.592118 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f4ba-5a38-407f-b314-399e8f0d4309 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:29.593095 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f4ba-5a38-407f-b314-399e8f0d4309 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:29.594025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f4ba-5a38-407f-b314-399e8f0d4309 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:29.595653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 134/282] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:29.597072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edafda-b7a1-490a-8f35-92fba89f4f1c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:29.601547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fe573-5a92-4ae7-a537-9e55472b7e9a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:29.602012 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fe573-5a92-4ae7-a537-9e55472b7e9a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:29.602429 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fe573-5a92-4ae7-a537-9e55472b7e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:29.628720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-446fe573-5a92-4ae7-a537-9e55472b7e9a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:29.630523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-446fe573-5a92-4ae7-a537-9e55472b7e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:29.763396 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edafda-b7a1-490a-8f35-92fba89f4f1c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:29.779630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:29.782946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 135/283] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:29.808706 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fe573-5a92-4ae7-a537-9e55472b7e9a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:29.813266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eacf9-1736-48bb-abf6-3539abe94c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:29.814630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eacf9-1736-48bb-abf6-3539abe94c1a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:29.814630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eacf9-1736-48bb-abf6-3539abe94c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:29.814630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eacf9-1736-48bb-abf6-3539abe94c1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:29.825466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57eacf9-1736-48bb-abf6-3539abe94c1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-a57eacf9-1736-48bb-abf6-3539abe94c1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:29.833922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:29.835556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:29.836467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 149/284] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:29.843654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eacf9-1736-48bb-abf6-3539abe94c1a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:29.951549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eacf9-1736-48bb-abf6-3539abe94c1a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:29.955499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:29.989273 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47945a2-8bc1-43d1-b88c-0314d9939851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:29.990045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47945a2-8bc1-43d1-b88c-0314d9939851 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:29.990432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47945a2-8bc1-43d1-b88c-0314d9939851 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:29.990718 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47945a2-8bc1-43d1-b88c-0314d9939851 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:29.991473 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47945a2-8bc1-43d1-b88c-0314d9939851 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:29.991787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47945a2-8bc1-43d1-b88c-0314d9939851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:29.993404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47945a2-8bc1-43d1-b88c-0314d9939851 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:29.999031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fe573-5a92-4ae7-a537-9e55472b7e9a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:30.003461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 150/285] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:30.009068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47945a2-8bc1-43d1-b88c-0314d9939851 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:30.028076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:30.037855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4b63c-9741-4115-90d7-86d6a9c246ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:30.038943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4b63c-9741-4115-90d7-86d6a9c246ec None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:30.039236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4b63c-9741-4115-90d7-86d6a9c246ec None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:30.039519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4b63c-9741-4115-90d7-86d6a9c246ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:30.052864 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4b63c-9741-4115-90d7-86d6a9c246ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0eb4b63c-9741-4115-90d7-86d6a9c246ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:30.063401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:30.065731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:30.067420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 136/286] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:41:30.073638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4b63c-9741-4115-90d7-86d6a9c246ec None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:30.129223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47945a2-8bc1-43d1-b88c-0314d9939851 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:30.166157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6657c2-6765-4506-bf77-79f0515d1915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:30.167031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6657c2-6765-4506-bf77-79f0515d1915 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:30.167319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6657c2-6765-4506-bf77-79f0515d1915 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:30.167591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6657c2-6765-4506-bf77-79f0515d1915 None admin] PATH_INFO: `/v3/users/70d8c50094824daaa570bd5a9b706278` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:30.168841 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6657c2-6765-4506-bf77-79f0515d1915 None admin] RBAC: Authorizing `identity:get_user(user_id=70d8c50094824daaa570bd5a9b706278)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:30.171456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6657c2-6765-4506-bf77-79f0515d1915 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:30.174084 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 151/287] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:30 2022] GET /identity/v3/users/70d8c50094824daaa570bd5a9b706278 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:30.179446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6657c2-6765-4506-bf77-79f0515d1915 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:30.279071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4b63c-9741-4115-90d7-86d6a9c246ec None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:30.317048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:30.326720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:30.350148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb76148-7c29-4a41-8950-76f4e970a8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:30.351237 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb76148-7c29-4a41-8950-76f4e970a8df None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:30.351731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb76148-7c29-4a41-8950-76f4e970a8df None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:30.352240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb76148-7c29-4a41-8950-76f4e970a8df None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:30.353065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6657c2-6765-4506-bf77-79f0515d1915 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:30.353489 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb76148-7c29-4a41-8950-76f4e970a8df None admin] RBAC: Adding query filter params (name=reader) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:30.354005 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb76148-7c29-4a41-8950-76f4e970a8df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:30.364710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb76148-7c29-4a41-8950-76f4e970a8df None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:30.381705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 137/288] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:30.389446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb76148-7c29-4a41-8950-76f4e970a8df None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:30.415021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84a1c6-93a7-4aab-a4ba-fc6f422bee02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:30.415734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84a1c6-93a7-4aab-a4ba-fc6f422bee02 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:30.416017 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84a1c6-93a7-4aab-a4ba-fc6f422bee02 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:30.416294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84a1c6-93a7-4aab-a4ba-fc6f422bee02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:30.503307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84a1c6-93a7-4aab-a4ba-fc6f422bee02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:30.503307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84a1c6-93a7-4aab-a4ba-fc6f422bee02 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:30.605081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:30.617268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 152/289] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 11:41:30.637963 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb76148-7c29-4a41-8950-76f4e970a8df None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:30.641259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84a1c6-93a7-4aab-a4ba-fc6f422bee02 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:30.722441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587d7e4-9dad-4941-b34d-88a21f0dacae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:30.723390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587d7e4-9dad-4941-b34d-88a21f0dacae None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:30.723676 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587d7e4-9dad-4941-b34d-88a21f0dacae None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:30.723955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6587d7e4-9dad-4941-b34d-88a21f0dacae None admin] PATH_INFO: `/v3/users/90db243021dc458bb98bd7d1760e2613` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:30.727355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587d7e4-9dad-4941-b34d-88a21f0dacae None admin] RBAC: Authorizing `identity:get_user(user_id=90db243021dc458bb98bd7d1760e2613)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:30.728832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6587d7e4-9dad-4941-b34d-88a21f0dacae None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:30.737060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 138/290] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:30 2022] GET /identity/v3/users/90db243021dc458bb98bd7d1760e2613 => generated 297 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:30.738992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587d7e4-9dad-4941-b34d-88a21f0dacae None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:30.836980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84a1c6-93a7-4aab-a4ba-fc6f422bee02 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:30.850745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:30.902648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6587d7e4-9dad-4941-b34d-88a21f0dacae None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:30.938028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc7be9-422b-460d-9d1a-b9c4320ae1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:30.948728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc7be9-422b-460d-9d1a-b9c4320ae1af None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:30.948728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc7be9-422b-460d-9d1a-b9c4320ae1af None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:30.948728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc7be9-422b-460d-9d1a-b9c4320ae1af None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:30.949833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2a4ce-2318-4a11-9ad9-c577be556c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:30.950873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2a4ce-2318-4a11-9ad9-c577be556c2e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:30.951718 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc7be9-422b-460d-9d1a-b9c4320ae1af None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:30.952155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2a4ce-2318-4a11-9ad9-c577be556c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:30.954345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc7be9-422b-460d-9d1a-b9c4320ae1af None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:30.954779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2a4ce-2318-4a11-9ad9-c577be556c2e None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:30.956179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2a4ce-2318-4a11-9ad9-c577be556c2e None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:30.957835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2a4ce-2318-4a11-9ad9-c577be556c2e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:30.968989 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 153/291] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:30 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:30.969364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 139/292] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:30 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:30.972912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc7be9-422b-460d-9d1a-b9c4320ae1af None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:30.973556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2a4ce-2318-4a11-9ad9-c577be556c2e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:31.093035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:31.148060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc7be9-422b-460d-9d1a-b9c4320ae1af None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:31.158842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2a4ce-2318-4a11-9ad9-c577be556c2e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:31.185764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd553709-6907-494d-8519-ef53e388924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:31.186794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd553709-6907-494d-8519-ef53e388924f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:31.187240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd553709-6907-494d-8519-ef53e388924f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:31.187534 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd553709-6907-494d-8519-ef53e388924f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:31.188385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd553709-6907-494d-8519-ef53e388924f None admin] RBAC: Adding query filter params (role.id=20703235b4c543bc9c51fb7c26da270d, scope.project.id=fcf4e9bfa6db4a5686e55857c6019793, user.id=70d8c50094824daaa570bd5a9b706278) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:31.188721 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd553709-6907-494d-8519-ef53e388924f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:31.190189 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd553709-6907-494d-8519-ef53e388924f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:31.197001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 154/293] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:30 2022] GET /identity/v3/role_assignments?user.id=70d8c50094824daaa570bd5a9b706278&scope.project.id=fcf4e9bfa6db4a5686e55857c6019793&role.id=20703235b4c543bc9c51fb7c26da270d => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:31.214712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:31.228425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ee6e3-f358-4d4b-9706-bd154383d893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:31.229378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ee6e3-f358-4d4b-9706-bd154383d893 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:31.229667 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ee6e3-f358-4d4b-9706-bd154383d893 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:31.229945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ee6e3-f358-4d4b-9706-bd154383d893 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793/users/90db243021dc458bb98bd7d1760e2613/roles/fe57e073b65947b199b35bd49fc6e30a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:31.232079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ee6e3-f358-4d4b-9706-bd154383d893 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf4e9bfa6db4a5686e55857c6019793, user_id=90db243021dc458bb98bd7d1760e2613, role_id=fe57e073b65947b199b35bd49fc6e30a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:31.233639 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ee6e3-f358-4d4b-9706-bd154383d893 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:31.250526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 140/294] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:41:30 2022] PUT /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793/users/90db243021dc458bb98bd7d1760e2613/roles/fe57e073b65947b199b35bd49fc6e30a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:41:31.329444 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5c50e-129b-4a1e-9cd0-d07b68768c22 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:31.329729 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5c50e-129b-4a1e-9cd0-d07b68768c22 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:31.330006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5c50e-129b-4a1e-9cd0-d07b68768c22 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:31.331396 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 155/295] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:31.337201 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327cbfc-e96a-4aea-968e-177f59ded3a6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:31.337484 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327cbfc-e96a-4aea-968e-177f59ded3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:31.337768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327cbfc-e96a-4aea-968e-177f59ded3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:31.361750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d327cbfc-e96a-4aea-968e-177f59ded3a6 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:31.363167 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d327cbfc-e96a-4aea-968e-177f59ded3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:31.456520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 141/296] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:31.481499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5c50e-129b-4a1e-9cd0-d07b68768c22 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:31.602146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5c50e-129b-4a1e-9cd0-d07b68768c22 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:31.614930 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:31.643647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936974d-e262-48dd-89ec-3c465e10540f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:31.644548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936974d-e262-48dd-89ec-3c465e10540f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:31.644873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936974d-e262-48dd-89ec-3c465e10540f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:31.645148 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936974d-e262-48dd-89ec-3c465e10540f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:31.658660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936974d-e262-48dd-89ec-3c465e10540f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-d936974d-e262-48dd-89ec-3c465e10540f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:31.666677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:31.667603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:31.668405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 156/297] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:31.677983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b23c80-710b-4fdd-bb1b-763c539f7dc1 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:31.678902 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b23c80-710b-4fdd-bb1b-763c539f7dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:31.679322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b23c80-710b-4fdd-bb1b-763c539f7dc1 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:31.680999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936974d-e262-48dd-89ec-3c465e10540f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:31.681376 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 142/298] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:31.689497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f5b59-a121-4b7b-92b0-79b1cd7cbd46 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:31.690155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f5b59-a121-4b7b-92b0-79b1cd7cbd46 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:31.690629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f5b59-a121-4b7b-92b0-79b1cd7cbd46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:31.714842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-232f5b59-a121-4b7b-92b0-79b1cd7cbd46 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:31.716591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-232f5b59-a121-4b7b-92b0-79b1cd7cbd46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:31.809005 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 143/299] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:31.826847 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936974d-e262-48dd-89ec-3c465e10540f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:31.832282 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f5b59-a121-4b7b-92b0-79b1cd7cbd46 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:31.864661 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb3cd5-8c5f-41b7-9d62-4e7871e72361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:31.865529 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb3cd5-8c5f-41b7-9d62-4e7871e72361 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:31.866037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb3cd5-8c5f-41b7-9d62-4e7871e72361 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:31.866519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb3cd5-8c5f-41b7-9d62-4e7871e72361 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:31.867679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb3cd5-8c5f-41b7-9d62-4e7871e72361 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:31.868059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb3cd5-8c5f-41b7-9d62-4e7871e72361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:31.869497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb3cd5-8c5f-41b7-9d62-4e7871e72361 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:31.877203 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:31.879848 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 157/300] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:31.886247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb3cd5-8c5f-41b7-9d62-4e7871e72361 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:31.924744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:31.961032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f5b59-a121-4b7b-92b0-79b1cd7cbd46 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:32.005137 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb3cd5-8c5f-41b7-9d62-4e7871e72361 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:32.013863 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cda7b-ce88-4c64-8a76-66dc65e42643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:32.014750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cda7b-ce88-4c64-8a76-66dc65e42643 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:32.015138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cda7b-ce88-4c64-8a76-66dc65e42643 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:32.015545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cda7b-ce88-4c64-8a76-66dc65e42643 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:32.017322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cda7b-ce88-4c64-8a76-66dc65e42643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:32.019546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cda7b-ce88-4c64-8a76-66dc65e42643 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:32.022275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 144/301] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:32.027191 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cda7b-ce88-4c64-8a76-66dc65e42643 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:32.045095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33a456-f535-45d0-b331-c643b9a1deb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:32.046871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33a456-f535-45d0-b331-c643b9a1deb0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:32.047466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33a456-f535-45d0-b331-c643b9a1deb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:32.048026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33a456-f535-45d0-b331-c643b9a1deb0 None admin] PATH_INFO: `/v3/users/4d8a097e2e2e4249b86924379e60b791` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:32.067912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33a456-f535-45d0-b331-c643b9a1deb0 None admin] RBAC: Authorizing `identity:get_user(user_id=4d8a097e2e2e4249b86924379e60b791)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:32.069519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33a456-f535-45d0-b331-c643b9a1deb0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:32.076112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 158/302] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:31 2022] GET /identity/v3/users/4d8a097e2e2e4249b86924379e60b791 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:32.077671 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c9538-2765-490a-9fa9-0f9d21488259 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:32.077950 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c9538-2765-490a-9fa9-0f9d21488259 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:32.078233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c9538-2765-490a-9fa9-0f9d21488259 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:32.079790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 159/303] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:32.080556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c9538-2765-490a-9fa9-0f9d21488259 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:32.152746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cda7b-ce88-4c64-8a76-66dc65e42643 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:32.164373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:32.183356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:32.192448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fcf20-463f-40ae-937d-3c60678e7459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:32.193253 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fcf20-463f-40ae-937d-3c60678e7459 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:32.193538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fcf20-463f-40ae-937d-3c60678e7459 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:32.193816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fcf20-463f-40ae-937d-3c60678e7459 None admin] PATH_INFO: `/v3/groups` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:32.194541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fcf20-463f-40ae-937d-3c60678e7459 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:32.196040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c9538-2765-490a-9fa9-0f9d21488259 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:32.196308 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fcf20-463f-40ae-937d-3c60678e7459 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:32.220358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 145/304] 10.222.0.67 () {60 vars in 1134 bytes} [Thu Jun 23 11:41:32 2022] POST /identity/v3/groups => generated 229 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:41:32.220358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a361a-8599-4486-8451-45e1d9631489 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:32.220358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a361a-8599-4486-8451-45e1d9631489 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:32.220358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a361a-8599-4486-8451-45e1d9631489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:32.238542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-786a361a-8599-4486-8451-45e1d9631489 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:32.239792 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-786a361a-8599-4486-8451-45e1d9631489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:32.247704 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7b8ca-f1ed-4b1d-a03f-fd75d843d0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:32.248387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7b8ca-f1ed-4b1d-a03f-fd75d843d0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:32.248735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7b8ca-f1ed-4b1d-a03f-fd75d843d0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:32.249028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7b8ca-f1ed-4b1d-a03f-fd75d843d0e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:32.251259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7b8ca-f1ed-4b1d-a03f-fd75d843d0e8 None admin] RBAC: Adding query filter params (role.id=fe57e073b65947b199b35bd49fc6e30a, user.id=4d8a097e2e2e4249b86924379e60b791, scope.system=all) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:32.251585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7b8ca-f1ed-4b1d-a03f-fd75d843d0e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:32.253867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7b8ca-f1ed-4b1d-a03f-fd75d843d0e8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:32.262992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 160/305] 10.222.0.67 () {56 vars in 1308 bytes} [Thu Jun 23 11:41:32 2022] GET /identity/v3/role_assignments?user.id=4d8a097e2e2e4249b86924379e60b791&scope.system=all&role.id=fe57e073b65947b199b35bd49fc6e30a => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:32.324613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 146/306] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:32.349774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7b8ca-f1ed-4b1d-a03f-fd75d843d0e8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:32.474003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7b8ca-f1ed-4b1d-a03f-fd75d843d0e8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:32.512515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8cacb-a402-4e53-8f92-4865fc7143cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:32.513979 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8cacb-a402-4e53-8f92-4865fc7143cf None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:32.514426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8cacb-a402-4e53-8f92-4865fc7143cf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:32.515429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e8cacb-a402-4e53-8f92-4865fc7143cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:32.532613 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e8cacb-a402-4e53-8f92-4865fc7143cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-78e8cacb-a402-4e53-8f92-4865fc7143cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:32.540436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:32.541370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:32.542171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 161/307] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:32.547902 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a361a-8599-4486-8451-45e1d9631489 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:32.653597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:32.662996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a361a-8599-4486-8451-45e1d9631489 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:32.701790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5986d3-f2c0-453b-ab93-8791f532dcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:32.703083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5986d3-f2c0-453b-ab93-8791f532dcdf None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:32.703447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5986d3-f2c0-453b-ab93-8791f532dcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:32.703784 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5986d3-f2c0-453b-ab93-8791f532dcdf None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:32.704605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5986d3-f2c0-453b-ab93-8791f532dcdf None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:32.705361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5986d3-f2c0-453b-ab93-8791f532dcdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:32.706943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5986d3-f2c0-453b-ab93-8791f532dcdf None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:32.721099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 147/308] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:32.724132 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8cacb-a402-4e53-8f92-4865fc7143cf None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:32.742121 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:32.852750 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e8cacb-a402-4e53-8f92-4865fc7143cf None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:32.911255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3672e015-ee40-4ebf-837a-30f029ca2018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:32.912663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672e015-ee40-4ebf-837a-30f029ca2018 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:32.913164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672e015-ee40-4ebf-837a-30f029ca2018 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:32.913640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3672e015-ee40-4ebf-837a-30f029ca2018 None admin] PATH_INFO: `/v3/users/72ce33c269464062a58d62a2246da7a1` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:32.915505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672e015-ee40-4ebf-837a-30f029ca2018 None admin] RBAC: Authorizing `identity:get_user(user_id=72ce33c269464062a58d62a2246da7a1)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:32.917922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3672e015-ee40-4ebf-837a-30f029ca2018 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:32.920695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 162/309] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:32 2022] GET /identity/v3/users/72ce33c269464062a58d62a2246da7a1 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:32.926119 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5986d3-f2c0-453b-ab93-8791f532dcdf None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:33.037640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5986d3-f2c0-453b-ab93-8791f532dcdf None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:33.073058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4dbb2-60d6-4900-ab99-5e8de91078fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:33.073925 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4dbb2-60d6-4900-ab99-5e8de91078fb None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:33.074204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4dbb2-60d6-4900-ab99-5e8de91078fb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:33.074479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4dbb2-60d6-4900-ab99-5e8de91078fb None admin] PATH_INFO: `/v3/system/users/72ce33c269464062a58d62a2246da7a1/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:33.076016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4dbb2-60d6-4900-ab99-5e8de91078fb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=72ce33c269464062a58d62a2246da7a1, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:33.077522 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4dbb2-60d6-4900-ab99-5e8de91078fb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:33.090166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 148/310] 10.222.0.67 () {58 vars in 1334 bytes} [Thu Jun 23 11:41:32 2022] PUT /identity/v3/system/users/72ce33c269464062a58d62a2246da7a1/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:41:33.554648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6290cbd-b5f0-47f6-bb1d-259b8d329a19 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:33.555051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6290cbd-b5f0-47f6-bb1d-259b8d329a19 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:33.555238 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6290cbd-b5f0-47f6-bb1d-259b8d329a19 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:33.556867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 163/311] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:33.562867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090ba8f-1826-4c03-9b67-9d8abc602147 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:33.563165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090ba8f-1826-4c03-9b67-9d8abc602147 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:33.563450 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090ba8f-1826-4c03-9b67-9d8abc602147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:33.580957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3090ba8f-1826-4c03-9b67-9d8abc602147 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:33.582169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3090ba8f-1826-4c03-9b67-9d8abc602147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:33.602916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:33.650778 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 149/312] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:33.675267 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6290cbd-b5f0-47f6-bb1d-259b8d329a19 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:33.761136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:33.781898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6290cbd-b5f0-47f6-bb1d-259b8d329a19 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:33.819853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aee87c-1e0b-4a82-adae-41595a85f415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:33.820697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aee87c-1e0b-4a82-adae-41595a85f415 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:33.820998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aee87c-1e0b-4a82-adae-41595a85f415 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:33.821282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aee87c-1e0b-4a82-adae-41595a85f415 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:33.832058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aee87c-1e0b-4a82-adae-41595a85f415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-25aee87c-1e0b-4a82-adae-41595a85f415 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:33.839878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:33.840785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:33.841584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 164/313] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:33.847511 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090ba8f-1826-4c03-9b67-9d8abc602147 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:33.951334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090ba8f-1826-4c03-9b67-9d8abc602147 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:33.985935 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872f1ab-2ff7-46de-8237-8941df2e53bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:33.986609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872f1ab-2ff7-46de-8237-8941df2e53bb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:33.986897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872f1ab-2ff7-46de-8237-8941df2e53bb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:33.987179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872f1ab-2ff7-46de-8237-8941df2e53bb None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:33.987937 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872f1ab-2ff7-46de-8237-8941df2e53bb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:33.988249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872f1ab-2ff7-46de-8237-8941df2e53bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:33.989678 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872f1ab-2ff7-46de-8237-8941df2e53bb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:34.000880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 150/314] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:34.006574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aee87c-1e0b-4a82-adae-41595a85f415 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:34.112835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aee87c-1e0b-4a82-adae-41595a85f415 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:34.147550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb24c3b-111e-4a54-9f7a-23e006b29f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:34.148432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb24c3b-111e-4a54-9f7a-23e006b29f84 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:34.148742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb24c3b-111e-4a54-9f7a-23e006b29f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:34.149027 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb24c3b-111e-4a54-9f7a-23e006b29f84 None admin] PATH_INFO: `/v3/users/90db243021dc458bb98bd7d1760e2613` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:34.150241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb24c3b-111e-4a54-9f7a-23e006b29f84 None admin] RBAC: Authorizing `identity:get_user(user_id=90db243021dc458bb98bd7d1760e2613)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:34.151689 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb24c3b-111e-4a54-9f7a-23e006b29f84 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:34.153677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 165/315] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:34 2022] GET /identity/v3/users/90db243021dc458bb98bd7d1760e2613 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:34.161767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872f1ab-2ff7-46de-8237-8941df2e53bb None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:34.181616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:34.270876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872f1ab-2ff7-46de-8237-8941df2e53bb None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:34.308073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cf627-3833-4c05-98c2-baeacd71d185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:34.308741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf627-3833-4c05-98c2-baeacd71d185 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:34.309030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf627-3833-4c05-98c2-baeacd71d185 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:34.309305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536cf627-3833-4c05-98c2-baeacd71d185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:34.381544 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cf627-3833-4c05-98c2-baeacd71d185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:34.382909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536cf627-3833-4c05-98c2-baeacd71d185 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:34.385592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc709f-790a-414d-9da8-9065d55ab64f None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:34.385894 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc709f-790a-414d-9da8-9065d55ab64f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:34.386185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc709f-790a-414d-9da8-9065d55ab64f None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:34.388609 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 166/316] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:34.393608 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c548ada-eff6-411e-86cd-bdbeef608c9f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:34.393905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c548ada-eff6-411e-86cd-bdbeef608c9f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:34.394192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c548ada-eff6-411e-86cd-bdbeef608c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:34.411213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6c548ada-eff6-411e-86cd-bdbeef608c9f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:34.412412 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6c548ada-eff6-411e-86cd-bdbeef608c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:34.415469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:34.418083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:34.483036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 167/317] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:34.486763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 151/318] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:34.496712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c548ada-eff6-411e-86cd-bdbeef608c9f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:34.505940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cf627-3833-4c05-98c2-baeacd71d185 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:34.604150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c548ada-eff6-411e-86cd-bdbeef608c9f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:34.616157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536cf627-3833-4c05-98c2-baeacd71d185 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:34.636268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:34.643479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814643f-bf60-4fa5-b53a-582196136240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:34.644473 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814643f-bf60-4fa5-b53a-582196136240 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:34.644801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814643f-bf60-4fa5-b53a-582196136240 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:34.645082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814643f-bf60-4fa5-b53a-582196136240 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:34.646266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814643f-bf60-4fa5-b53a-582196136240 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:34.647700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814643f-bf60-4fa5-b53a-582196136240 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:34.649437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 168/319] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:34 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:34.655342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a23a9-be0f-40d4-9829-a0409669dd15 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:34.655498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a889e-fc7e-44d5-ad17-54cf05c5391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:34.655696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a23a9-be0f-40d4-9829-a0409669dd15 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:34.655948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a23a9-be0f-40d4-9829-a0409669dd15 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:34.656138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a889e-fc7e-44d5-ad17-54cf05c5391d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:34.656419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a889e-fc7e-44d5-ad17-54cf05c5391d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:34.656987 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a889e-fc7e-44d5-ad17-54cf05c5391d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:34.657550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 169/320] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:34.660981 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a23a9-be0f-40d4-9829-a0409669dd15 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:34.672563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651a889e-fc7e-44d5-ad17-54cf05c5391d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:34.674206 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-651a889e-fc7e-44d5-ad17-54cf05c5391d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:34.681506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:34.682406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 152/321] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:41:34.688879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3f554-85a9-44da-94e9-67bc27b57cb7 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:34.689762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3f554-85a9-44da-94e9-67bc27b57cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:34.689762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3f554-85a9-44da-94e9-67bc27b57cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:34.703642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-00b3f554-85a9-44da-94e9-67bc27b57cb7 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:34.704883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-00b3f554-85a9-44da-94e9-67bc27b57cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:34.775914 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a23a9-be0f-40d4-9829-a0409669dd15 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:34.784709 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 153/322] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:34.784709 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3f554-85a9-44da-94e9-67bc27b57cb7 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:34.821818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37c0f7-e010-4630-9633-995996a85588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:34.822776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37c0f7-e010-4630-9633-995996a85588 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:34.823065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37c0f7-e010-4630-9633-995996a85588 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:34.823343 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37c0f7-e010-4630-9633-995996a85588 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:34.824183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37c0f7-e010-4630-9633-995996a85588 None admin] RBAC: Adding query filter params (role.id=fe57e073b65947b199b35bd49fc6e30a, scope.project.id=fcf4e9bfa6db4a5686e55857c6019793, user.id=90db243021dc458bb98bd7d1760e2613) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:34.824495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37c0f7-e010-4630-9633-995996a85588 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:34.826181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37c0f7-e010-4630-9633-995996a85588 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:34.840702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 170/323] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:41:34 2022] GET /identity/v3/role_assignments?user.id=90db243021dc458bb98bd7d1760e2613&scope.project.id=fcf4e9bfa6db4a5686e55857c6019793&role.id=fe57e073b65947b199b35bd49fc6e30a => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:34.840702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37c0f7-e010-4630-9633-995996a85588 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:34.924006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3f554-85a9-44da-94e9-67bc27b57cb7 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:34.927275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:34.939668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:34.971837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37c0f7-e010-4630-9633-995996a85588 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:34.977905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6fe28e-9ff6-43f6-9eed-763b7c06773a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:34.978592 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fe28e-9ff6-43f6-9eed-763b7c06773a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:34.978877 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fe28e-9ff6-43f6-9eed-763b7c06773a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:34.979153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6fe28e-9ff6-43f6-9eed-763b7c06773a None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:34.980600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6fe28e-9ff6-43f6-9eed-763b7c06773a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:34.980953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6fe28e-9ff6-43f6-9eed-763b7c06773a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:34.983117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6fe28e-9ff6-43f6-9eed-763b7c06773a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:34.996713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 154/324] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:35.004256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6fe28e-9ff6-43f6-9eed-763b7c06773a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:35.020197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21a228-3ba6-44dd-9142-74f6f094164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:35.020982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21a228-3ba6-44dd-9142-74f6f094164c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:35.021268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21a228-3ba6-44dd-9142-74f6f094164c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:35.021545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21a228-3ba6-44dd-9142-74f6f094164c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:35.023804 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21a228-3ba6-44dd-9142-74f6f094164c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:35.025217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21a228-3ba6-44dd-9142-74f6f094164c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:35.027302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 171/325] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:35.032873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21a228-3ba6-44dd-9142-74f6f094164c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:35.135839 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6fe28e-9ff6-43f6-9eed-763b7c06773a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:35.162698 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21a228-3ba6-44dd-9142-74f6f094164c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:35.171921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1acaa2-796a-4e6e-bc14-8bb403b6c201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:35.172822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1acaa2-796a-4e6e-bc14-8bb403b6c201 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:35.173113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1acaa2-796a-4e6e-bc14-8bb403b6c201 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:35.173391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1acaa2-796a-4e6e-bc14-8bb403b6c201 None admin] PATH_INFO: `/v3/users/4d8a097e2e2e4249b86924379e60b791` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:35.174668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1acaa2-796a-4e6e-bc14-8bb403b6c201 None admin] RBAC: Authorizing `identity:get_user(user_id=4d8a097e2e2e4249b86924379e60b791)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:35.176057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1acaa2-796a-4e6e-bc14-8bb403b6c201 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:35.177748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 155/326] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:35 2022] GET /identity/v3/users/4d8a097e2e2e4249b86924379e60b791 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:35.183445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1acaa2-796a-4e6e-bc14-8bb403b6c201 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:35.201878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049f4d3-13e5-4fd8-856e-3133c1e63b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:35.203013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049f4d3-13e5-4fd8-856e-3133c1e63b81 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:35.203578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049f4d3-13e5-4fd8-856e-3133c1e63b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:35.204326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049f4d3-13e5-4fd8-856e-3133c1e63b81 None admin] PATH_INFO: `/v3/groups` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:35.205515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049f4d3-13e5-4fd8-856e-3133c1e63b81 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:35.207186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049f4d3-13e5-4fd8-856e-3133c1e63b81 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:35.221935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:35.229049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 172/327] 10.222.0.67 () {60 vars in 1134 bytes} [Thu Jun 23 11:41:35 2022] POST /identity/v3/groups => generated 226 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:41:35.238539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:35.267125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec4723-2bbc-45d7-9519-0a7df411f9d8 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:35.267426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec4723-2bbc-45d7-9519-0a7df411f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:35.267706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec4723-2bbc-45d7-9519-0a7df411f9d8 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:35.269332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 173/328] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:41:35.274714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327cc35-5d0e-4378-943f-e8bb300d0977 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:35.275011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327cc35-5d0e-4378-943f-e8bb300d0977 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:35.275298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5327cc35-5d0e-4378-943f-e8bb300d0977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:35.297931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5327cc35-5d0e-4378-943f-e8bb300d0977 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:35.299213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5327cc35-5d0e-4378-943f-e8bb300d0977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:35.312367 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1acaa2-796a-4e6e-bc14-8bb403b6c201 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:35.350685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12d67a-82dd-4ddf-ba6c-36ac2be1bf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:35.351563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12d67a-82dd-4ddf-ba6c-36ac2be1bf0f None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:35.351853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12d67a-82dd-4ddf-ba6c-36ac2be1bf0f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:35.352130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12d67a-82dd-4ddf-ba6c-36ac2be1bf0f None admin] PATH_INFO: `/v3/system/users/4d8a097e2e2e4249b86924379e60b791/roles/fe57e073b65947b199b35bd49fc6e30a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:35.353786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12d67a-82dd-4ddf-ba6c-36ac2be1bf0f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4d8a097e2e2e4249b86924379e60b791, role_id=fe57e073b65947b199b35bd49fc6e30a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:35.355174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a12d67a-82dd-4ddf-ba6c-36ac2be1bf0f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:35.365262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 156/329] 10.222.0.67 () {58 vars in 1334 bytes} [Thu Jun 23 11:41:35 2022] PUT /identity/v3/system/users/4d8a097e2e2e4249b86924379e60b791/roles/fe57e073b65947b199b35bd49fc6e30a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:41:35.384147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 174/330] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:35.408098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12d67a-82dd-4ddf-ba6c-36ac2be1bf0f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:35.526984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12d67a-82dd-4ddf-ba6c-36ac2be1bf0f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:35.528577 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:35.565595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02189405-d945-408c-a2d0-b82acc7b5e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:35.566484 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02189405-d945-408c-a2d0-b82acc7b5e6f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:35.566831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02189405-d945-408c-a2d0-b82acc7b5e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:35.567172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02189405-d945-408c-a2d0-b82acc7b5e6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:35.579429 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02189405-d945-408c-a2d0-b82acc7b5e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-02189405-d945-408c-a2d0-b82acc7b5e6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:35.587362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:35.588473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:35.589501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:35.589501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:35.589501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:35.589501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:35.589501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:35.589501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:35.589501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:35.589501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:35.589501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:35.589501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:35.589501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:35.589501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:35.590750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 157/331] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:35.597083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327cc35-5d0e-4378-943f-e8bb300d0977 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:35.705513 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5327cc35-5d0e-4378-943f-e8bb300d0977 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:35.738502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:35.744486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757887d6-e0c9-4a2e-b119-736fb7725719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:35.747919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757887d6-e0c9-4a2e-b119-736fb7725719 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:35.748200 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757887d6-e0c9-4a2e-b119-736fb7725719 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:35.748886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757887d6-e0c9-4a2e-b119-736fb7725719 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:35.749685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757887d6-e0c9-4a2e-b119-736fb7725719 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:35.749995 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757887d6-e0c9-4a2e-b119-736fb7725719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:35.751385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757887d6-e0c9-4a2e-b119-736fb7725719 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:35.761189 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 175/332] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:35.767155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02189405-d945-408c-a2d0-b82acc7b5e6f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:35.873780 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02189405-d945-408c-a2d0-b82acc7b5e6f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:35.914887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74154089-f74c-4584-a26a-84d94ec6fe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:35.915971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74154089-f74c-4584-a26a-84d94ec6fe6c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:35.916257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74154089-f74c-4584-a26a-84d94ec6fe6c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:35.916538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74154089-f74c-4584-a26a-84d94ec6fe6c None admin] PATH_INFO: `/v3/users/72ce33c269464062a58d62a2246da7a1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:35.917879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74154089-f74c-4584-a26a-84d94ec6fe6c None admin] RBAC: Authorizing `identity:get_user(user_id=72ce33c269464062a58d62a2246da7a1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:35.919452 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74154089-f74c-4584-a26a-84d94ec6fe6c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:35.921647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 158/333] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:35 2022] GET /identity/v3/users/72ce33c269464062a58d62a2246da7a1 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:35.927353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757887d6-e0c9-4a2e-b119-736fb7725719 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:36.049245 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757887d6-e0c9-4a2e-b119-736fb7725719 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:36.090022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661994e5-57ad-4725-aaa8-d6352abca860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:36.090745 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661994e5-57ad-4725-aaa8-d6352abca860 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:36.091026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661994e5-57ad-4725-aaa8-d6352abca860 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:36.091306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661994e5-57ad-4725-aaa8-d6352abca860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:36.092198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661994e5-57ad-4725-aaa8-d6352abca860 None admin] RBAC: Adding query filter params (role.id=20703235b4c543bc9c51fb7c26da270d, user.id=72ce33c269464062a58d62a2246da7a1, scope.system=all) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:36.092517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661994e5-57ad-4725-aaa8-d6352abca860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:36.094059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661994e5-57ad-4725-aaa8-d6352abca860 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:36.106908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 176/334] 10.222.0.67 () {56 vars in 1308 bytes} [Thu Jun 23 11:41:35 2022] GET /identity/v3/role_assignments?user.id=72ce33c269464062a58d62a2246da7a1&scope.system=all&role.id=20703235b4c543bc9c51fb7c26da270d => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:37.662171 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a4000-6c5e-42b9-8667-a53de0c381a5 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:37.662673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a4000-6c5e-42b9-8667-a53de0c381a5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:37.662748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a4000-6c5e-42b9-8667-a53de0c381a5 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:37.664417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 159/335] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:37.669862 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41b961-4af6-46c7-84f7-1c1fd725b1ce None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:37.670251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41b961-4af6-46c7-84f7-1c1fd725b1ce None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:37.670416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41b961-4af6-46c7-84f7-1c1fd725b1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:37.690040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ea41b961-4af6-46c7-84f7-1c1fd725b1ce None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:37.691254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ea41b961-4af6-46c7-84f7-1c1fd725b1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:37.748892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:37.778374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 177/336] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:37.825399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a4000-6c5e-42b9-8667-a53de0c381a5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:37.867594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:37.950126 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a4000-6c5e-42b9-8667-a53de0c381a5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:37.989779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa370c-1f95-4b62-ae9e-38f644eda9ee None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:37.990416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa370c-1f95-4b62-ae9e-38f644eda9ee None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:37.990901 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa370c-1f95-4b62-ae9e-38f644eda9ee None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:37.992819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 178/337] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:37.998064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd3514-ba1e-48c9-a42e-de7a866d0fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:37.999180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd3514-ba1e-48c9-a42e-de7a866d0fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:38.000198 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd3514-ba1e-48c9-a42e-de7a866d0fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:38.000698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd3514-ba1e-48c9-a42e-de7a866d0fb4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:38.006164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7877d6f-a3f5-4e7b-b7f2-3003480d5381 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:38.006454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7877d6f-a3f5-4e7b-b7f2-3003480d5381 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:38.006739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7877d6f-a3f5-4e7b-b7f2-3003480d5381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:38.029603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd3514-ba1e-48c9-a42e-de7a866d0fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-34cd3514-ba1e-48c9-a42e-de7a866d0fb4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:38.037675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:38.038573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:38.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:38.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:38.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:38.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:38.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:38.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:38.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:38.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:38.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:38.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:38.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 11:41:38.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:38.040009 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e7877d6f-a3f5-4e7b-b7f2-3003480d5381 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:38.045313 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e7877d6f-a3f5-4e7b-b7f2-3003480d5381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:38.049284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 160/338] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:38.050820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd3514-ba1e-48c9-a42e-de7a866d0fb4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:38.177716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 179/339] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:38.179080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a063a14-4e89-4432-9ed5-2dc80e43d18d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:38.179368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a063a14-4e89-4432-9ed5-2dc80e43d18d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:38.179648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a063a14-4e89-4432-9ed5-2dc80e43d18d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:38.186011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 180/340] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:38.208424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a24d3-a75d-45f1-9b5b-76add29debce None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:38.208719 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a24d3-a75d-45f1-9b5b-76add29debce None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:38.208951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a24d3-a75d-45f1-9b5b-76add29debce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:38.239011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-302a24d3-a75d-45f1-9b5b-76add29debce None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:38.240218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-302a24d3-a75d-45f1-9b5b-76add29debce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:38.241012 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:38.258787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd3514-ba1e-48c9-a42e-de7a866d0fb4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:38.302411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe682b-910c-43bd-8871-03e85c05fc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:38.303079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe682b-910c-43bd-8871-03e85c05fc2b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:38.303360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe682b-910c-43bd-8871-03e85c05fc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:38.303636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbe682b-910c-43bd-8871-03e85c05fc2b None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:38.304389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe682b-910c-43bd-8871-03e85c05fc2b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:38.304776 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe682b-910c-43bd-8871-03e85c05fc2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:38.306318 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbe682b-910c-43bd-8871-03e85c05fc2b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:38.317220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 161/341] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:38.319081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe682b-910c-43bd-8871-03e85c05fc2b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:38.406541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 181/342] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:38.407696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a24d3-a75d-45f1-9b5b-76add29debce None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:38.464017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbe682b-910c-43bd-8871-03e85c05fc2b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:38.497532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:38.527311 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba970cc-32ff-4a7d-a146-d6fb03aba8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:38.528115 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba970cc-32ff-4a7d-a146-d6fb03aba8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:38.528407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba970cc-32ff-4a7d-a146-d6fb03aba8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:38.529161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba970cc-32ff-4a7d-a146-d6fb03aba8b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:38.550675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba970cc-32ff-4a7d-a146-d6fb03aba8b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-9ba970cc-32ff-4a7d-a146-d6fb03aba8b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:38.565869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:38.566815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:38.567647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:38.567647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:38.567647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:38.567647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:38.567647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:38.567647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:38.567647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:38.567647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:38.567647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:38.567647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:38.567647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:38.567647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:38.568837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 162/343] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:38.574335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba970cc-32ff-4a7d-a146-d6fb03aba8b2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:38.596357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:38.605946 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a24d3-a75d-45f1-9b5b-76add29debce None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:38.674655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc926b-d87c-4261-b473-2c22bdeaec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:38.674655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc926b-d87c-4261-b473-2c22bdeaec1d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:38.674655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc926b-d87c-4261-b473-2c22bdeaec1d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:38.674655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc926b-d87c-4261-b473-2c22bdeaec1d None admin] PATH_INFO: `/v3/users/4d8a097e2e2e4249b86924379e60b791` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:38.677797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc926b-d87c-4261-b473-2c22bdeaec1d None admin] RBAC: Authorizing `identity:get_user(user_id=4d8a097e2e2e4249b86924379e60b791)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:38.679258 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc926b-d87c-4261-b473-2c22bdeaec1d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:38.681158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 182/344] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:41:38 2022] GET /identity/v3/users/4d8a097e2e2e4249b86924379e60b791 => generated 293 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:38.681709 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc926b-d87c-4261-b473-2c22bdeaec1d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:38.764433 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba970cc-32ff-4a7d-a146-d6fb03aba8b2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:38.830917 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd43f3-4c81-4ff9-b79d-186747262c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:38.831741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd43f3-4c81-4ff9-b79d-186747262c7a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:38.832023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd43f3-4c81-4ff9-b79d-186747262c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:38.832298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd43f3-4c81-4ff9-b79d-186747262c7a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:38.852335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd43f3-4c81-4ff9-b79d-186747262c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-e6cd43f3-4c81-4ff9-b79d-186747262c7a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:38.864028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:38.865001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:38.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:38.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:38.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:38.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:38.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:38.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:38.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:38.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:38.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:38.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:38.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:38.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:38.867098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 163/345] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:38.867968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd43f3-4c81-4ff9-b79d-186747262c7a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:38.894856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc926b-d87c-4261-b473-2c22bdeaec1d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:38.925132 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:38.951909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd55f8b-42d9-44f1-9a48-348bbdb40ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:38.952824 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd55f8b-42d9-44f1-9a48-348bbdb40ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:38.953217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd55f8b-42d9-44f1-9a48-348bbdb40ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:38.953601 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd55f8b-42d9-44f1-9a48-348bbdb40ff3 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:38.954467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd55f8b-42d9-44f1-9a48-348bbdb40ff3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:38.954787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd55f8b-42d9-44f1-9a48-348bbdb40ff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:38.956180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd55f8b-42d9-44f1-9a48-348bbdb40ff3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:38.970969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 183/346] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:38.971789 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd55f8b-42d9-44f1-9a48-348bbdb40ff3 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:38.981477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:39.025583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd43f3-4c81-4ff9-b79d-186747262c7a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:39.074446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcd250-ff3f-4f1e-9b97-142f5bbf36a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:39.075566 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcd250-ff3f-4f1e-9b97-142f5bbf36a2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:39.076129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcd250-ff3f-4f1e-9b97-142f5bbf36a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:39.076417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcd250-ff3f-4f1e-9b97-142f5bbf36a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:39.077342 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcd250-ff3f-4f1e-9b97-142f5bbf36a2 None admin] RBAC: Adding query filter params (role.id=fe57e073b65947b199b35bd49fc6e30a, user.id=4d8a097e2e2e4249b86924379e60b791, scope.system=all) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:39.077665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcd250-ff3f-4f1e-9b97-142f5bbf36a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:39.079380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bcd250-ff3f-4f1e-9b97-142f5bbf36a2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:39.098234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 164/347] 10.222.0.67 () {56 vars in 1308 bytes} [Thu Jun 23 11:41:38 2022] GET /identity/v3/role_assignments?user.id=4d8a097e2e2e4249b86924379e60b791&scope.system=all&role.id=fe57e073b65947b199b35bd49fc6e30a => generated 510 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:39.099223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd55f8b-42d9-44f1-9a48-348bbdb40ff3 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:39.099438 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27ed90-14a2-4e1f-8681-ab786d0169be None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:39.099653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27ed90-14a2-4e1f-8681-ab786d0169be None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:39.099939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e27ed90-14a2-4e1f-8681-ab786d0169be None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:39.102896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 165/348] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:39.103714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27ed90-14a2-4e1f-8681-ab786d0169be None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:39.208261 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff73cad1-c4d4-40cd-956e-722c3c5f25cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:39.208984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73cad1-c4d4-40cd-956e-722c3c5f25cf None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:39.209274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73cad1-c4d4-40cd-956e-722c3c5f25cf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:39.209556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73cad1-c4d4-40cd-956e-722c3c5f25cf None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:39.210328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73cad1-c4d4-40cd-956e-722c3c5f25cf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:39.210644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73cad1-c4d4-40cd-956e-722c3c5f25cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:39.212063 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff73cad1-c4d4-40cd-956e-722c3c5f25cf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:39.235938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 184/349] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:39.237415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbb903-e308-4848-be85-951abcedcba1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:39.240942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbb903-e308-4848-be85-951abcedcba1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:39.241300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbb903-e308-4848-be85-951abcedcba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:39.246014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e27ed90-14a2-4e1f-8681-ab786d0169be None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:39.272991 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-35bbb903-e308-4848-be85-951abcedcba1 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:39.275374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-35bbb903-e308-4848-be85-951abcedcba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:39.283960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:39.300341 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647e0e4-06f3-46bf-9c10-b07b98667c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:39.301025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647e0e4-06f3-46bf-9c10-b07b98667c2f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:39.301313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647e0e4-06f3-46bf-9c10-b07b98667c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:39.301813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647e0e4-06f3-46bf-9c10-b07b98667c2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:39.324854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:39.385509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 185/350] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:39.386794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbb903-e308-4848-be85-951abcedcba1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:39.397172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647e0e4-06f3-46bf-9c10-b07b98667c2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:39.398654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647e0e4-06f3-46bf-9c10-b07b98667c2f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:39.519788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 166/351] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:39.521209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647e0e4-06f3-46bf-9c10-b07b98667c2f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:39.521837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbb903-e308-4848-be85-951abcedcba1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:39.555671 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:39.560359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb94ccb-04db-4c4e-b692-4b471c7e7c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:39.561315 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb94ccb-04db-4c4e-b692-4b471c7e7c0d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:39.562337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb94ccb-04db-4c4e-b692-4b471c7e7c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:39.562803 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb94ccb-04db-4c4e-b692-4b471c7e7c0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:39.595242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:39.633356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647e0e4-06f3-46bf-9c10-b07b98667c2f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:39.643443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb94ccb-04db-4c4e-b692-4b471c7e7c0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:39.644957 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb94ccb-04db-4c4e-b692-4b471c7e7c0d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:39.682902 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2bd26-92bc-4c91-963f-dde166c94d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:39.683909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2bd26-92bc-4c91-963f-dde166c94d33 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:39.684203 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2bd26-92bc-4c91-963f-dde166c94d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:39.684499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed2bd26-92bc-4c91-963f-dde166c94d33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:39.697838 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed2bd26-92bc-4c91-963f-dde166c94d33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-2ed2bd26-92bc-4c91-963f-dde166c94d33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:39.707294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:39.708407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:39.709343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:39.709343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:39.709343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:39.709343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:39.709343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:39.709343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:39.709343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:39.709343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:39.709343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:39.709343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:39.709343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:39.709343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:39.710317 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 167/352] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:39.710317 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2bd26-92bc-4c91-963f-dde166c94d33 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:39.788465 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 186/353] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:39.789713 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7997af-4184-4b55-bdc1-f77bb47cb2b6 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:39.790005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7997af-4184-4b55-bdc1-f77bb47cb2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:39.790284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7997af-4184-4b55-bdc1-f77bb47cb2b6 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:39.791716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 187/354] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:39.792240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7997af-4184-4b55-bdc1-f77bb47cb2b6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:39.825680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:39.839131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed2bd26-92bc-4c91-963f-dde166c94d33 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:39.871391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:39.882078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbfd2c-d5d3-4bd0-8108-c184fa3984e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:39.882960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbfd2c-d5d3-4bd0-8108-c184fa3984e5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:39.883248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbfd2c-d5d3-4bd0-8108-c184fa3984e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:39.883541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbfd2c-d5d3-4bd0-8108-c184fa3984e5 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:39.886177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbfd2c-d5d3-4bd0-8108-c184fa3984e5 None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:39.887324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbfd2c-d5d3-4bd0-8108-c184fa3984e5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:39.889283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 168/355] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:39 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:39.889910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbfd2c-d5d3-4bd0-8108-c184fa3984e5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:39.944483 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7997af-4184-4b55-bdc1-f77bb47cb2b6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:39.995842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632c8d1-552b-408a-be8f-d9b057990710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:39.996793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632c8d1-552b-408a-be8f-d9b057990710 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:39.997236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632c8d1-552b-408a-be8f-d9b057990710 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:39.997519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632c8d1-552b-408a-be8f-d9b057990710 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:39.998281 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632c8d1-552b-408a-be8f-d9b057990710 None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:39.998595 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632c8d1-552b-408a-be8f-d9b057990710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:40.000548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632c8d1-552b-408a-be8f-d9b057990710 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:40.011580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 188/356] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:40.013073 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b0417-6534-4d3b-a821-d951b61a3cf8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:40.013361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b0417-6534-4d3b-a821-d951b61a3cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:40.013663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b0417-6534-4d3b-a821-d951b61a3cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:40.036077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-496b0417-6534-4d3b-a821-d951b61a3cf8 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:40.037330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-496b0417-6534-4d3b-a821-d951b61a3cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:40.039147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbfd2c-d5d3-4bd0-8108-c184fa3984e5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:40.089250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f33ed-cc10-49be-85fe-b4afb1153651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:40.090037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f33ed-cc10-49be-85fe-b4afb1153651 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:40.090327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f33ed-cc10-49be-85fe-b4afb1153651 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:40.090609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f33ed-cc10-49be-85fe-b4afb1153651 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:40.092799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f33ed-cc10-49be-85fe-b4afb1153651 None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:40.094293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f33ed-cc10-49be-85fe-b4afb1153651 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:40.096181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 169/357] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:39 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:40.097782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f33ed-cc10-49be-85fe-b4afb1153651 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:40.118202 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 189/358] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:40.119150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b0417-6534-4d3b-a821-d951b61a3cf8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:40.129870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:40.181506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:40.223083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f33ed-cc10-49be-85fe-b4afb1153651 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:40.241350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b0417-6534-4d3b-a821-d951b61a3cf8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:40.269653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f2676-4055-445e-b68a-38585ddf5586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:40.270748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f2676-4055-445e-b68a-38585ddf5586 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:40.271183 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f2676-4055-445e-b68a-38585ddf5586 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:40.271460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f2676-4055-445e-b68a-38585ddf5586 None admin] PATH_INFO: `/v3/groups/32a5071d1aef4422bbd2629d8bb75103` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:40.279229 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f2676-4055-445e-b68a-38585ddf5586 None admin] RBAC: Authorizing `identity:get_group(group_id=32a5071d1aef4422bbd2629d8bb75103)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:40.282519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f2676-4055-445e-b68a-38585ddf5586 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:40.282819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 170/359] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:40 2022] GET /identity/v3/groups/32a5071d1aef4422bbd2629d8bb75103 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:40.283677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a41401-5f80-44a8-b5f0-a473820e8df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:40.284342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a41401-5f80-44a8-b5f0-a473820e8df8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:40.284820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a41401-5f80-44a8-b5f0-a473820e8df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:40.285052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a41401-5f80-44a8-b5f0-a473820e8df8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:40.286164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacb7a3-d889-4a91-a894-a1ef7957aaaf None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:40.286459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacb7a3-d889-4a91-a894-a1ef7957aaaf None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:40.286741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacb7a3-d889-4a91-a894-a1ef7957aaaf None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:40.288422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 171/360] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:40.290056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacb7a3-d889-4a91-a894-a1ef7957aaaf None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:40.367228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a41401-5f80-44a8-b5f0-a473820e8df8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:40.368771 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a41401-5f80-44a8-b5f0-a473820e8df8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:40.398615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:40.411911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacb7a3-d889-4a91-a894-a1ef7957aaaf None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:40.433244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:40.458505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41038a-637b-445e-8277-565fccfa33b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:40.459370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41038a-637b-445e-8277-565fccfa33b9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:40.459658 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41038a-637b-445e-8277-565fccfa33b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:40.459941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41038a-637b-445e-8277-565fccfa33b9 None admin] PATH_INFO: `/v3/groups/32a5071d1aef4422bbd2629d8bb75103` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:40.461162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41038a-637b-445e-8277-565fccfa33b9 None admin] RBAC: Authorizing `identity:get_group(group_id=32a5071d1aef4422bbd2629d8bb75103)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:40.462618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41038a-637b-445e-8277-565fccfa33b9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:40.466861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 172/361] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:40 2022] GET /identity/v3/groups/32a5071d1aef4422bbd2629d8bb75103 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:40.466861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41038a-637b-445e-8277-565fccfa33b9 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:40.489615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 190/362] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:40.490478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a41401-5f80-44a8-b5f0-a473820e8df8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:40.597949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41038a-637b-445e-8277-565fccfa33b9 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:40.612498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a41401-5f80-44a8-b5f0-a473820e8df8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:40.639735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d9fa73-5ac7-4e5a-b725-50c3e9680f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:40.640477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9fa73-5ac7-4e5a-b725-50c3e9680f3f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:40.640840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9fa73-5ac7-4e5a-b725-50c3e9680f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:40.641123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d9fa73-5ac7-4e5a-b725-50c3e9680f3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:40.650779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904dbc20-2b7f-4b58-8fb4-59df1811df8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:40.651434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dbc20-2b7f-4b58-8fb4-59df1811df8d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:40.651722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dbc20-2b7f-4b58-8fb4-59df1811df8d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:40.652006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904dbc20-2b7f-4b58-8fb4-59df1811df8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:40.652147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d9fa73-5ac7-4e5a-b725-50c3e9680f3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:40.652875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904dbc20-2b7f-4b58-8fb4-59df1811df8d None admin] RBAC: Adding query filter params (group.id=32a5071d1aef4422bbd2629d8bb75103, role.id=20703235b4c543bc9c51fb7c26da270d, scope.project.id=ea37fde41a0c43a790da11fb742b5490) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:40.653195 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904dbc20-2b7f-4b58-8fb4-59df1811df8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:40.654639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904dbc20-2b7f-4b58-8fb4-59df1811df8d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-38d9fa73-5ac7-4e5a-b725-50c3e9680f3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:40.661330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:40.662264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:40.663070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 173/363] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:40.664115 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109adac7-7a45-40bd-868c-7136b5c5680a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:40.664405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109adac7-7a45-40bd-868c-7136b5c5680a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:40.664746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109adac7-7a45-40bd-868c-7136b5c5680a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:40.669427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 191/364] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 23 11:41:40 2022] GET /identity/v3/role_assignments?group.id=32a5071d1aef4422bbd2629d8bb75103&scope.project.id=ea37fde41a0c43a790da11fb742b5490&role.id=20703235b4c543bc9c51fb7c26da270d => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:40.670363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904dbc20-2b7f-4b58-8fb4-59df1811df8d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:40.686654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-109adac7-7a45-40bd-868c-7136b5c5680a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:40.688410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-109adac7-7a45-40bd-868c-7136b5c5680a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:40.699158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:40.711229 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:40.787956 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 174/365] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:40.788486 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109adac7-7a45-40bd-868c-7136b5c5680a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:40.818618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904dbc20-2b7f-4b58-8fb4-59df1811df8d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:40.870826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb981b3-c76c-4f29-ab23-44dbd9ac9f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:40.871492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb981b3-c76c-4f29-ab23-44dbd9ac9f20 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:40.871776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb981b3-c76c-4f29-ab23-44dbd9ac9f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:40.872055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb981b3-c76c-4f29-ab23-44dbd9ac9f20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:40.873394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb981b3-c76c-4f29-ab23-44dbd9ac9f20 None admin] RBAC: Adding query filter params (group.id=32a5071d1aef4422bbd2629d8bb75103, role.id=2d0e0231af5d4e51b80de6bb65e8b02b, scope.project.id=ea37fde41a0c43a790da11fb742b5490) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:40.873394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb981b3-c76c-4f29-ab23-44dbd9ac9f20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:40.874713 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb981b3-c76c-4f29-ab23-44dbd9ac9f20 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:40.888814 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 192/366] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 23 11:41:40 2022] GET /identity/v3/role_assignments?group.id=32a5071d1aef4422bbd2629d8bb75103&scope.project.id=ea37fde41a0c43a790da11fb742b5490&role.id=2d0e0231af5d4e51b80de6bb65e8b02b => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:40.891821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb981b3-c76c-4f29-ab23-44dbd9ac9f20 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:40.913824 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109adac7-7a45-40bd-868c-7136b5c5680a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:40.957383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a1ad6-e6e3-447b-95c8-5134eb8c84b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:40.958265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a1ad6-e6e3-447b-95c8-5134eb8c84b9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:40.958543 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a1ad6-e6e3-447b-95c8-5134eb8c84b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:40.958815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a1ad6-e6e3-447b-95c8-5134eb8c84b9 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:40.960049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a1ad6-e6e3-447b-95c8-5134eb8c84b9 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:40.961833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a1ad6-e6e3-447b-95c8-5134eb8c84b9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:40.963601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 175/367] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:40 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:40.964383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a1ad6-e6e3-447b-95c8-5134eb8c84b9 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:40.992188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:41.026335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:41.053125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb981b3-c76c-4f29-ab23-44dbd9ac9f20 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:41.100233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe706e3-8413-4224-ade3-7dc047f6a9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:41.101121 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe706e3-8413-4224-ade3-7dc047f6a9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:41.101231 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe706e3-8413-4224-ade3-7dc047f6a9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:41.101509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe706e3-8413-4224-ade3-7dc047f6a9a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:41.102270 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe706e3-8413-4224-ade3-7dc047f6a9a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:41.102585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe706e3-8413-4224-ade3-7dc047f6a9a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:41.103948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe706e3-8413-4224-ade3-7dc047f6a9a9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:41.105000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a1ad6-e6e3-447b-95c8-5134eb8c84b9 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:41.113704 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 193/368] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:41.114661 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe706e3-8413-4224-ade3-7dc047f6a9a9 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:41.152567 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edeb2d-a7e9-4f5a-9094-3edb87b39f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:41.153350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edeb2d-a7e9-4f5a-9094-3edb87b39f54 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:41.153638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edeb2d-a7e9-4f5a-9094-3edb87b39f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:41.153920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01edeb2d-a7e9-4f5a-9094-3edb87b39f54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:41.173366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01edeb2d-a7e9-4f5a-9094-3edb87b39f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-01edeb2d-a7e9-4f5a-9094-3edb87b39f54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:41.181822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:41.182717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:41.183500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:41.183500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:41.183500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:41.183500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:41.183500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:41.183500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:41.183500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:41.183500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:41.183500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:41.183500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:41.183500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:41.183500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:41.184092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 176/369] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:41:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:41.185162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edeb2d-a7e9-4f5a-9094-3edb87b39f54 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:41.240417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe706e3-8413-4224-ade3-7dc047f6a9a9 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:41.276542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:41.305512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed28b6-3a65-4f01-99bd-a7a13bfd5135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:41.305750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01edeb2d-a7e9-4f5a-9094-3edb87b39f54 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:41.306892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed28b6-3a65-4f01-99bd-a7a13bfd5135 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:41.307614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed28b6-3a65-4f01-99bd-a7a13bfd5135 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:41.308911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed28b6-3a65-4f01-99bd-a7a13bfd5135 None admin] PATH_INFO: `/v3/groups/32a5071d1aef4422bbd2629d8bb75103` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:41.313495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed28b6-3a65-4f01-99bd-a7a13bfd5135 None admin] RBAC: Authorizing `identity:get_group(group_id=32a5071d1aef4422bbd2629d8bb75103)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:41.315801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed28b6-3a65-4f01-99bd-a7a13bfd5135 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:41.318677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 194/370] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:41 2022] GET /identity/v3/groups/32a5071d1aef4422bbd2629d8bb75103 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:41.320237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed28b6-3a65-4f01-99bd-a7a13bfd5135 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:41.348352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:41.361781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddf7ca-1796-444d-be3f-9e59098aa8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:41.362610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddf7ca-1796-444d-be3f-9e59098aa8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:41.363069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddf7ca-1796-444d-be3f-9e59098aa8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:41.363405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddf7ca-1796-444d-be3f-9e59098aa8e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:41.445088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddf7ca-1796-444d-be3f-9e59098aa8e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:41.446535 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddf7ca-1796-444d-be3f-9e59098aa8e1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:41.447499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed28b6-3a65-4f01-99bd-a7a13bfd5135 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:41.487067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b252d7f-d783-4333-8d42-e6e549036a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:41.488099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b252d7f-d783-4333-8d42-e6e549036a59 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:41.488603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b252d7f-d783-4333-8d42-e6e549036a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:41.489109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b252d7f-d783-4333-8d42-e6e549036a59 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:41.490099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b252d7f-d783-4333-8d42-e6e549036a59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:41.490629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b252d7f-d783-4333-8d42-e6e549036a59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:41.492416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b252d7f-d783-4333-8d42-e6e549036a59 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:41.510764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 195/371] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:41.513678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b252d7f-d783-4333-8d42-e6e549036a59 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:41.553259 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:41.586896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 177/372] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:41.588937 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddf7ca-1796-444d-be3f-9e59098aa8e1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:41.640793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:41.685426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b252d7f-d783-4333-8d42-e6e549036a59 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:41.737178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7f06b-3039-4fd5-8853-f6e1fd0aa7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:41.738026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7f06b-3039-4fd5-8853-f6e1fd0aa7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:41.738408 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7f06b-3039-4fd5-8853-f6e1fd0aa7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:41.738781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7f06b-3039-4fd5-8853-f6e1fd0aa7b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:41.739793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7f06b-3039-4fd5-8853-f6e1fd0aa7b1 None admin] RBAC: Adding query filter params (group.id=32a5071d1aef4422bbd2629d8bb75103, role.id=20703235b4c543bc9c51fb7c26da270d, scope.project.id=fcf4e9bfa6db4a5686e55857c6019793) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:41.740740 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7f06b-3039-4fd5-8853-f6e1fd0aa7b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:41.742431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7f06b-3039-4fd5-8853-f6e1fd0aa7b1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:41.753578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 196/373] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 23 11:41:41 2022] GET /identity/v3/role_assignments?group.id=32a5071d1aef4422bbd2629d8bb75103&scope.project.id=fcf4e9bfa6db4a5686e55857c6019793&role.id=20703235b4c543bc9c51fb7c26da270d => generated 260 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:41.754663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7f06b-3039-4fd5-8853-f6e1fd0aa7b1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:41.769074 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddf7ca-1796-444d-be3f-9e59098aa8e1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:41.845660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803d3a6-485c-4ee8-9cd7-72a1cc0d8a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:41.846354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803d3a6-485c-4ee8-9cd7-72a1cc0d8a5d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:41.846637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803d3a6-485c-4ee8-9cd7-72a1cc0d8a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:41.846915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803d3a6-485c-4ee8-9cd7-72a1cc0d8a5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:41.939659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:41.971969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803d3a6-485c-4ee8-9cd7-72a1cc0d8a5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:41.973982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f803d3a6-485c-4ee8-9cd7-72a1cc0d8a5d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:42.002856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7f06b-3039-4fd5-8853-f6e1fd0aa7b1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:42.076185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0b390-1195-4f66-8415-bb2a27c86ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:42.077525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0b390-1195-4f66-8415-bb2a27c86ffb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:42.078047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0b390-1195-4f66-8415-bb2a27c86ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:42.079523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0b390-1195-4f66-8415-bb2a27c86ffb None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:42.081149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0b390-1195-4f66-8415-bb2a27c86ffb None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:42.082818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0b390-1195-4f66-8415-bb2a27c86ffb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:42.085183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 197/374] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:41 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:42.090497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0b390-1195-4f66-8415-bb2a27c86ffb None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:42.102830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:42.123592 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 178/375] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:42.129840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803d3a6-485c-4ee8-9cd7-72a1cc0d8a5d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:42.213213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0b390-1195-4f66-8415-bb2a27c86ffb None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:42.221238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:42.242241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803d3a6-485c-4ee8-9cd7-72a1cc0d8a5d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:42.256518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a71c1-31a8-48e6-b584-05164c1d40b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:42.257862 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a71c1-31a8-48e6-b584-05164c1d40b5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:42.258363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a71c1-31a8-48e6-b584-05164c1d40b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:42.258861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a71c1-31a8-48e6-b584-05164c1d40b5 None admin] PATH_INFO: `/v3/groups/32a5071d1aef4422bbd2629d8bb75103` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:42.260464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a71c1-31a8-48e6-b584-05164c1d40b5 None admin] RBAC: Authorizing `identity:get_group(group_id=32a5071d1aef4422bbd2629d8bb75103)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:42.262158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a71c1-31a8-48e6-b584-05164c1d40b5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:42.264489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 198/376] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:42 2022] GET /identity/v3/groups/32a5071d1aef4422bbd2629d8bb75103 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:42.273790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a71c1-31a8-48e6-b584-05164c1d40b5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:42.292700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef2fc5-a548-44ba-a20e-dd2db030a5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:42.292700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef2fc5-a548-44ba-a20e-dd2db030a5aa None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:42.292700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef2fc5-a548-44ba-a20e-dd2db030a5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:42.292700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef2fc5-a548-44ba-a20e-dd2db030a5aa None admin] PATH_INFO: `/v3/projects/1e503e3101124c66a9726948b602dc87` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:42.293132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef2fc5-a548-44ba-a20e-dd2db030a5aa None admin] RBAC: Authorizing `identity:get_project(project_id=1e503e3101124c66a9726948b602dc87)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:42.293132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef2fc5-a548-44ba-a20e-dd2db030a5aa None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:42.295235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 179/377] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:42 2022] GET /identity/v3/projects/1e503e3101124c66a9726948b602dc87 => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:42.301281 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef2fc5-a548-44ba-a20e-dd2db030a5aa None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:42.425488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:42.495328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a71c1-31a8-48e6-b584-05164c1d40b5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:42.521093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef2fc5-a548-44ba-a20e-dd2db030a5aa None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:42.625722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1c67de-27cc-45db-b5f4-1bc386a46f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:42.626850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c67de-27cc-45db-b5f4-1bc386a46f9c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:42.627391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c67de-27cc-45db-b5f4-1bc386a46f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:42.627902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1c67de-27cc-45db-b5f4-1bc386a46f9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:42.632498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c67de-27cc-45db-b5f4-1bc386a46f9c None admin] RBAC: Adding query filter params (group.id=32a5071d1aef4422bbd2629d8bb75103, role.id=2d0e0231af5d4e51b80de6bb65e8b02b, scope.project.id=fcf4e9bfa6db4a5686e55857c6019793) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:42.632894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51fb6a-0d83-4bee-b401-6348204a3b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:42.633686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c67de-27cc-45db-b5f4-1bc386a46f9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:42.633805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51fb6a-0d83-4bee-b401-6348204a3b5f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:42.634007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51fb6a-0d83-4bee-b401-6348204a3b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:42.634290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e51fb6a-0d83-4bee-b401-6348204a3b5f None admin] PATH_INFO: `/v3/groups/71eb78847cac4374b5b516fdfa03dcac` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:42.636095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1c67de-27cc-45db-b5f4-1bc386a46f9c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:42.641209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51fb6a-0d83-4bee-b401-6348204a3b5f None admin] RBAC: Authorizing `identity:get_group(group_id=71eb78847cac4374b5b516fdfa03dcac)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:42.642736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e51fb6a-0d83-4bee-b401-6348204a3b5f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:42.645522 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 180/378] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:42 2022] GET /identity/v3/groups/71eb78847cac4374b5b516fdfa03dcac => generated 229 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:42.646557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 199/379] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 23 11:41:42 2022] GET /identity/v3/role_assignments?group.id=32a5071d1aef4422bbd2629d8bb75103&scope.project.id=fcf4e9bfa6db4a5686e55857c6019793&role.id=2d0e0231af5d4e51b80de6bb65e8b02b => generated 260 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:42.659091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51fb6a-0d83-4bee-b401-6348204a3b5f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:42.681559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:42.801061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e51fb6a-0d83-4bee-b401-6348204a3b5f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:42.840164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13cec2-c487-490d-876f-c4c910880066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:42.840926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13cec2-c487-490d-876f-c4c910880066 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:42.841219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13cec2-c487-490d-876f-c4c910880066 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:42.841504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13cec2-c487-490d-876f-c4c910880066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:42.842365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13cec2-c487-490d-876f-c4c910880066 None admin] RBAC: Adding query filter params (group.id=71eb78847cac4374b5b516fdfa03dcac, role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=1e503e3101124c66a9726948b602dc87) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:42.842685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13cec2-c487-490d-876f-c4c910880066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:42.844791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13cec2-c487-490d-876f-c4c910880066 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:42.852074 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 181/380] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 23 11:41:42 2022] GET /identity/v3/role_assignments?group.id=71eb78847cac4374b5b516fdfa03dcac&scope.project.id=1e503e3101124c66a9726948b602dc87&role.id=569707378794437b93d168fc8bb9cbc3 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:43.246006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773f3ab-e7be-44dd-936a-fdc48ffd7e09 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:43.247029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773f3ab-e7be-44dd-936a-fdc48ffd7e09 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:43.247537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773f3ab-e7be-44dd-936a-fdc48ffd7e09 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:43.250048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 200/381] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:43.256054 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89b622-cc6e-4d0f-9313-6fea8df24a15 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:43.257008 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89b622-cc6e-4d0f-9313-6fea8df24a15 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:43.257503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89b622-cc6e-4d0f-9313-6fea8df24a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:43.282022 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3d89b622-cc6e-4d0f-9313-6fea8df24a15 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:43.283592 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3d89b622-cc6e-4d0f-9313-6fea8df24a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:43.299497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb618343-653b-4f55-b25a-fe831e78faf6 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:43.299835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb618343-653b-4f55-b25a-fe831e78faf6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:43.300058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb618343-653b-4f55-b25a-fe831e78faf6 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:43.304702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 201/382] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:43.307731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc25bd-4173-45f0-a7fa-d7292e7a180a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:43.308028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc25bd-4173-45f0-a7fa-d7292e7a180a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:43.308329 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc25bd-4173-45f0-a7fa-d7292e7a180a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:43.336330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4bdc25bd-4173-45f0-a7fa-d7292e7a180a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:43.339677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4bdc25bd-4173-45f0-a7fa-d7292e7a180a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:43.410872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 182/383] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:43.465539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 202/384] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:43.475697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89b622-cc6e-4d0f-9313-6fea8df24a15 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:43.479635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:43.489451 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc25bd-4173-45f0-a7fa-d7292e7a180a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:43.574174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:43.594703 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc25bd-4173-45f0-a7fa-d7292e7a180a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:43.622028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89b622-cc6e-4d0f-9313-6fea8df24a15 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:43.630839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb36b9e-c32c-477e-91c9-46e03b955afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:43.631662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb36b9e-c32c-477e-91c9-46e03b955afd None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:43.631949 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb36b9e-c32c-477e-91c9-46e03b955afd None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:43.632230 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb36b9e-c32c-477e-91c9-46e03b955afd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:43.643139 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb36b9e-c32c-477e-91c9-46e03b955afd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-beb36b9e-c32c-477e-91c9-46e03b955afd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:43.651506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:43.652459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:43.653342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:43.653342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:43.653342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:43.653342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:43.653342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:43.653342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:43.653342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:43.653342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:43.653342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:43.653342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:43.653342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:43.653342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:43.657028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 203/385] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:41:43.660284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb36b9e-c32c-477e-91c9-46e03b955afd None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:43.660818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59181e-9da9-40cf-a537-176a6ce5e583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:43.661633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59181e-9da9-40cf-a537-176a6ce5e583 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:43.661920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59181e-9da9-40cf-a537-176a6ce5e583 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:43.662205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59181e-9da9-40cf-a537-176a6ce5e583 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:43.680712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf59181e-9da9-40cf-a537-176a6ce5e583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-bf59181e-9da9-40cf-a537-176a6ce5e583 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:43.683256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:43.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:43.684960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 183/386] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:43.690598 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59181e-9da9-40cf-a537-176a6ce5e583 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:43.763542 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb36b9e-c32c-477e-91c9-46e03b955afd None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:43.791175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:43.803957 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438285f-85f5-4dc8-91b4-42a19af3bd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:43.804634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438285f-85f5-4dc8-91b4-42a19af3bd92 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:43.805647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438285f-85f5-4dc8-91b4-42a19af3bd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:43.805931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438285f-85f5-4dc8-91b4-42a19af3bd92 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:43.806697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438285f-85f5-4dc8-91b4-42a19af3bd92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:43.807012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438285f-85f5-4dc8-91b4-42a19af3bd92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:43.808398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438285f-85f5-4dc8-91b4-42a19af3bd92 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:43.812264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:43.829028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 204/387] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:43.832614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438285f-85f5-4dc8-91b4-42a19af3bd92 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:43.942814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59181e-9da9-40cf-a537-176a6ce5e583 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:43.951089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438285f-85f5-4dc8-91b4-42a19af3bd92 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:43.993112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9be88-76b5-4ffe-b5a0-1af498a682af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:43.994727 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9be88-76b5-4ffe-b5a0-1af498a682af None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:43.995164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9be88-76b5-4ffe-b5a0-1af498a682af None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:43.995601 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9be88-76b5-4ffe-b5a0-1af498a682af None admin] PATH_INFO: `/v3/groups/32a5071d1aef4422bbd2629d8bb75103` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:43.997048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9be88-76b5-4ffe-b5a0-1af498a682af None admin] RBAC: Authorizing `identity:get_group(group_id=32a5071d1aef4422bbd2629d8bb75103)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:43.998883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9be88-76b5-4ffe-b5a0-1af498a682af None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:44.000801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 205/388] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:43 2022] GET /identity/v3/groups/32a5071d1aef4422bbd2629d8bb75103 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:44.006705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9be88-76b5-4ffe-b5a0-1af498a682af None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:44.063976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1b961-aee8-4a8f-845c-c93aa97eb2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:44.064972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1b961-aee8-4a8f-845c-c93aa97eb2db None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:44.065417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1b961-aee8-4a8f-845c-c93aa97eb2db None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:44.065849 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1b961-aee8-4a8f-845c-c93aa97eb2db None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:44.066764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1b961-aee8-4a8f-845c-c93aa97eb2db None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:44.067235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1b961-aee8-4a8f-845c-c93aa97eb2db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:44.068800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1b961-aee8-4a8f-845c-c93aa97eb2db None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:44.092509 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 184/389] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:44.095553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1b961-aee8-4a8f-845c-c93aa97eb2db None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:44.159058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:44.220585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1b961-aee8-4a8f-845c-c93aa97eb2db None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:44.246537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:44.259107 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c589e8c-9e39-4859-b748-1e9d7e49d2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:44.260788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c589e8c-9e39-4859-b748-1e9d7e49d2ca None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:44.261257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c589e8c-9e39-4859-b748-1e9d7e49d2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:44.261729 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c589e8c-9e39-4859-b748-1e9d7e49d2ca None admin] PATH_INFO: `/v3/groups/32a5071d1aef4422bbd2629d8bb75103` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:44.263890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c589e8c-9e39-4859-b748-1e9d7e49d2ca None admin] RBAC: Authorizing `identity:get_group(group_id=32a5071d1aef4422bbd2629d8bb75103)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:44.266235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c589e8c-9e39-4859-b748-1e9d7e49d2ca None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:44.268685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 185/390] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:44 2022] GET /identity/v3/groups/32a5071d1aef4422bbd2629d8bb75103 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:44.273881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c589e8c-9e39-4859-b748-1e9d7e49d2ca None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:44.322718 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9be88-76b5-4ffe-b5a0-1af498a682af None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:44.404400 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49e465-8ae3-4898-9955-f7379709fe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:44.405866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49e465-8ae3-4898-9955-f7379709fe7c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:44.406347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49e465-8ae3-4898-9955-f7379709fe7c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:44.407065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49e465-8ae3-4898-9955-f7379709fe7c None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:44.408508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49e465-8ae3-4898-9955-f7379709fe7c None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:44.410929 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49e465-8ae3-4898-9955-f7379709fe7c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:44.411871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c589e8c-9e39-4859-b748-1e9d7e49d2ca None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:44.413205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 206/391] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:44 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:44.420225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49e465-8ae3-4898-9955-f7379709fe7c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:44.490044 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08c23d-1efc-4ba1-9729-1ef4bfe9fbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:44.490936 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08c23d-1efc-4ba1-9729-1ef4bfe9fbb3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:44.491224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08c23d-1efc-4ba1-9729-1ef4bfe9fbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:44.491505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f08c23d-1efc-4ba1-9729-1ef4bfe9fbb3 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:44.497553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08c23d-1efc-4ba1-9729-1ef4bfe9fbb3 None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:44.499104 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f08c23d-1efc-4ba1-9729-1ef4bfe9fbb3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:44.501836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 186/392] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:44 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:44.509687 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08c23d-1efc-4ba1-9729-1ef4bfe9fbb3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:44.558518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49e465-8ae3-4898-9955-f7379709fe7c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:44.598044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:44.614102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d45c35-d38a-4f33-9d11-0b78cdbc63cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:44.615338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d45c35-d38a-4f33-9d11-0b78cdbc63cb None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:44.615804 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d45c35-d38a-4f33-9d11-0b78cdbc63cb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:44.616266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d45c35-d38a-4f33-9d11-0b78cdbc63cb None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490/groups/32a5071d1aef4422bbd2629d8bb75103/roles/2d0e0231af5d4e51b80de6bb65e8b02b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:44.620060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d45c35-d38a-4f33-9d11-0b78cdbc63cb None admin] RBAC: Authorizing `identity:create_grant(project_id=ea37fde41a0c43a790da11fb742b5490, group_id=32a5071d1aef4422bbd2629d8bb75103, role_id=2d0e0231af5d4e51b80de6bb65e8b02b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:44.622123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d45c35-d38a-4f33-9d11-0b78cdbc63cb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:44.648797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 207/393] 10.222.0.67 () {58 vars in 1442 bytes} [Thu Jun 23 11:41:44 2022] PUT /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490/groups/32a5071d1aef4422bbd2629d8bb75103/roles/2d0e0231af5d4e51b80de6bb65e8b02b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:41:44.685292 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:44.696825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9830-9852-4c33-9339-b1b4d24b7b76 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:44.697141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9830-9852-4c33-9339-b1b4d24b7b76 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:44.697392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a9830-9852-4c33-9339-b1b4d24b7b76 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:44.699456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 208/394] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:41:44.706168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca5869-15e6-4117-99fe-bbe2dec191e8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:44.706497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca5869-15e6-4117-99fe-bbe2dec191e8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:44.707106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca5869-15e6-4117-99fe-bbe2dec191e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:44.725921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-acca5869-15e6-4117-99fe-bbe2dec191e8 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:44.727795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-acca5869-15e6-4117-99fe-bbe2dec191e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:44.790106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f08c23d-1efc-4ba1-9729-1ef4bfe9fbb3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:44.867743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 209/395] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:44.869888 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caaeb65-9f05-4740-8a38-97c1bd605b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:44.871802 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caaeb65-9f05-4740-8a38-97c1bd605b27 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:44.871802 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caaeb65-9f05-4740-8a38-97c1bd605b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:44.871802 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caaeb65-9f05-4740-8a38-97c1bd605b27 None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490/groups/32a5071d1aef4422bbd2629d8bb75103/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:44.873350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caaeb65-9f05-4740-8a38-97c1bd605b27 None admin] RBAC: Authorizing `identity:create_grant(project_id=ea37fde41a0c43a790da11fb742b5490, group_id=32a5071d1aef4422bbd2629d8bb75103, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:44.874999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caaeb65-9f05-4740-8a38-97c1bd605b27 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:44.893770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca5869-15e6-4117-99fe-bbe2dec191e8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:44.898740 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 187/396] 10.222.0.67 () {58 vars in 1442 bytes} [Thu Jun 23 11:41:44 2022] PUT /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490/groups/32a5071d1aef4422bbd2629d8bb75103/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:41:45.024478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca5869-15e6-4117-99fe-bbe2dec191e8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:45.047342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:45.076427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8acbe6-9aa5-4b81-89e7-f64d6a70dd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:45.077321 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8acbe6-9aa5-4b81-89e7-f64d6a70dd93 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:45.077618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8acbe6-9aa5-4b81-89e7-f64d6a70dd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:45.077894 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8acbe6-9aa5-4b81-89e7-f64d6a70dd93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:45.095522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8acbe6-9aa5-4b81-89e7-f64d6a70dd93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-ef8acbe6-9aa5-4b81-89e7-f64d6a70dd93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:45.108028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:45.109624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:45.110433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 210/397] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:41:45.119454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caaeb65-9f05-4740-8a38-97c1bd605b27 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:45.242026 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caaeb65-9f05-4740-8a38-97c1bd605b27 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:45.266458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:45.295162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df81ffe-d666-4335-a688-81171eff4e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:45.296050 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df81ffe-d666-4335-a688-81171eff4e93 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:45.296523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df81ffe-d666-4335-a688-81171eff4e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:45.297036 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df81ffe-d666-4335-a688-81171eff4e93 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:45.298458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df81ffe-d666-4335-a688-81171eff4e93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:45.299026 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df81ffe-d666-4335-a688-81171eff4e93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:45.301403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df81ffe-d666-4335-a688-81171eff4e93 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:45.317713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 188/398] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:45.323091 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8acbe6-9aa5-4b81-89e7-f64d6a70dd93 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:45.437827 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e277456-a125-4998-964a-52fa431a07d7 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:45.438164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e277456-a125-4998-964a-52fa431a07d7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:45.438379 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e277456-a125-4998-964a-52fa431a07d7 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:45.440142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 189/399] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:45.445743 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d6d9d-c322-426e-a896-a32b52b12372 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:45.446045 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d6d9d-c322-426e-a896-a32b52b12372 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:45.446304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d6d9d-c322-426e-a896-a32b52b12372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:45.466418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8acbe6-9aa5-4b81-89e7-f64d6a70dd93 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:45.472197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-fe5d6d9d-c322-426e-a896-a32b52b12372 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:45.473823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fe5d6d9d-c322-426e-a896-a32b52b12372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:45.502384 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52564218-f2a5-4421-88bd-d338bd7b4719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:45.503247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52564218-f2a5-4421-88bd-d338bd7b4719 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:45.503535 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52564218-f2a5-4421-88bd-d338bd7b4719 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:45.503814 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52564218-f2a5-4421-88bd-d338bd7b4719 None admin] PATH_INFO: `/v3/groups/32a5071d1aef4422bbd2629d8bb75103` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:45.505020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52564218-f2a5-4421-88bd-d338bd7b4719 None admin] RBAC: Authorizing `identity:get_group(group_id=32a5071d1aef4422bbd2629d8bb75103)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:45.506474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52564218-f2a5-4421-88bd-d338bd7b4719 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:45.508517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 211/400] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:45 2022] GET /identity/v3/groups/32a5071d1aef4422bbd2629d8bb75103 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:45.509850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87683f7-471b-493b-aeb5-fea923b6b277 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:45.510143 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87683f7-471b-493b-aeb5-fea923b6b277 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:45.510421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87683f7-471b-493b-aeb5-fea923b6b277 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:45.511779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 212/401] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:45.514680 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87683f7-471b-493b-aeb5-fea923b6b277 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:45.575893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 190/402] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:45.577537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0fdbf-96e6-49f5-9295-c3532425c03c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:45.577537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0fdbf-96e6-49f5-9295-c3532425c03c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:45.577835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0fdbf-96e6-49f5-9295-c3532425c03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:45.591029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:45.595880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8ae0fdbf-96e6-49f5-9295-c3532425c03c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:45.597391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8ae0fdbf-96e6-49f5-9295-c3532425c03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:45.690323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87683f7-471b-493b-aeb5-fea923b6b277 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:45.690897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 191/403] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:45.697492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0fdbf-96e6-49f5-9295-c3532425c03c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:45.735684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af59e62-457b-4577-ad64-ab28218c4e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:45.737004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af59e62-457b-4577-ad64-ab28218c4e15 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:45.737561 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af59e62-457b-4577-ad64-ab28218c4e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:45.737841 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af59e62-457b-4577-ad64-ab28218c4e15 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:45.739301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af59e62-457b-4577-ad64-ab28218c4e15 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:45.741403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af59e62-457b-4577-ad64-ab28218c4e15 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:45.744325 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 213/404] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:45 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:45.749372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af59e62-457b-4577-ad64-ab28218c4e15 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:45.765216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:45.825590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0fdbf-96e6-49f5-9295-c3532425c03c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:45.852615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af59e62-457b-4577-ad64-ab28218c4e15 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:45.861850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1119d8-30b1-4ea6-b664-fb768536696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:45.862941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1119d8-30b1-4ea6-b664-fb768536696c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:45.863487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1119d8-30b1-4ea6-b664-fb768536696c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:45.863960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1119d8-30b1-4ea6-b664-fb768536696c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:45.875016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1119d8-30b1-4ea6-b664-fb768536696c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:45.881104 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-2d1119d8-30b1-4ea6-b664-fb768536696c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:45.883411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:45.884559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:45.885692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:45.885692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:45.885692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:45.885692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:45.885692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:45.885692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:45.885692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:45.885692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:45.885692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:45.885692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:45.885692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:45.885692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:45.892449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafae31-11d0-4e91-aa64-9a1bb940931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:45.892449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafae31-11d0-4e91-aa64-9a1bb940931a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:45.892449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafae31-11d0-4e91-aa64-9a1bb940931a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:45.892449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafae31-11d0-4e91-aa64-9a1bb940931a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:45.893271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 192/405] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:41:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:45.894322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1119d8-30b1-4ea6-b664-fb768536696c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:45.902501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eafae31-11d0-4e91-aa64-9a1bb940931a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-1eafae31-11d0-4e91-aa64-9a1bb940931a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:45.911793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:45.912972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:45.913823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:45.913823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:45.913823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:45.913823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:45.913823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:45.913823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:45.913823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:45.913823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:45.913823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:45.913823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:45.913823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:45.913823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:45.918157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 214/406] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:45.919257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafae31-11d0-4e91-aa64-9a1bb940931a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:46.013985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1119d8-30b1-4ea6-b664-fb768536696c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:46.016487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:46.030892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafae31-11d0-4e91-aa64-9a1bb940931a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:46.053850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22b294-9267-4ccd-8f89-0ecb331af199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:46.055088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22b294-9267-4ccd-8f89-0ecb331af199 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:46.055564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22b294-9267-4ccd-8f89-0ecb331af199 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:46.056066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22b294-9267-4ccd-8f89-0ecb331af199 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793/groups/32a5071d1aef4422bbd2629d8bb75103/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:46.059133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22b294-9267-4ccd-8f89-0ecb331af199 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf4e9bfa6db4a5686e55857c6019793, group_id=32a5071d1aef4422bbd2629d8bb75103, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:46.062116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22b294-9267-4ccd-8f89-0ecb331af199 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:46.067342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd8cca-5728-4e40-9bea-338920e01fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:46.067980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd8cca-5728-4e40-9bea-338920e01fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:46.069013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd8cca-5728-4e40-9bea-338920e01fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:46.069424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd8cca-5728-4e40-9bea-338920e01fb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:46.070022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd8cca-5728-4e40-9bea-338920e01fb6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:46.070376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd8cca-5728-4e40-9bea-338920e01fb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:46.071704 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd8cca-5728-4e40-9bea-338920e01fb6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:46.084744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 215/408] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:46.084744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd8cca-5728-4e40-9bea-338920e01fb6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:46.087996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 193/408] 10.222.0.67 () {58 vars in 1442 bytes} [Thu Jun 23 11:41:45 2022] PUT /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793/groups/32a5071d1aef4422bbd2629d8bb75103/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:41:46.092186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22b294-9267-4ccd-8f89-0ecb331af199 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:46.120564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:46.194202 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd8cca-5728-4e40-9bea-338920e01fb6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:46.232307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56270962-d90c-468f-9152-c2c36fe87f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:46.233494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56270962-d90c-468f-9152-c2c36fe87f14 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:46.233997 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56270962-d90c-468f-9152-c2c36fe87f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:46.234484 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56270962-d90c-468f-9152-c2c36fe87f14 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:46.235531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56270962-d90c-468f-9152-c2c36fe87f14 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:46.236099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56270962-d90c-468f-9152-c2c36fe87f14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:46.237748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56270962-d90c-468f-9152-c2c36fe87f14 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:46.251568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 216/409] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:46.258053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56270962-d90c-468f-9152-c2c36fe87f14 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:46.286814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22b294-9267-4ccd-8f89-0ecb331af199 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:46.369419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5847b-4f63-4672-b017-3761eb64a75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:46.370705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5847b-4f63-4672-b017-3761eb64a75e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:46.371202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5847b-4f63-4672-b017-3761eb64a75e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:46.371738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5847b-4f63-4672-b017-3761eb64a75e None admin] PATH_INFO: `/v3/groups/71eb78847cac4374b5b516fdfa03dcac` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:46.373261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5847b-4f63-4672-b017-3761eb64a75e None admin] RBAC: Authorizing `identity:get_group(group_id=71eb78847cac4374b5b516fdfa03dcac)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:46.375072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5847b-4f63-4672-b017-3761eb64a75e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:46.378446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 194/410] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:46 2022] GET /identity/v3/groups/71eb78847cac4374b5b516fdfa03dcac => generated 229 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:46.384966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5847b-4f63-4672-b017-3761eb64a75e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:46.407422 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:46.417123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56270962-d90c-468f-9152-c2c36fe87f14 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:46.455228 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:46.483020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05098e02-06a2-497e-a60c-e3d9c856b02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:46.485049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05098e02-06a2-497e-a60c-e3d9c856b02c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:46.487068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05098e02-06a2-497e-a60c-e3d9c856b02c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:46.487573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05098e02-06a2-497e-a60c-e3d9c856b02c None admin] PATH_INFO: `/v3/groups/32a5071d1aef4422bbd2629d8bb75103` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:46.490456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05098e02-06a2-497e-a60c-e3d9c856b02c None admin] RBAC: Authorizing `identity:get_group(group_id=32a5071d1aef4422bbd2629d8bb75103)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:46.492340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05098e02-06a2-497e-a60c-e3d9c856b02c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:46.500351 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 217/411] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:46 2022] GET /identity/v3/groups/32a5071d1aef4422bbd2629d8bb75103 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:46.504657 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05098e02-06a2-497e-a60c-e3d9c856b02c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:46.599070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5847b-4f63-4672-b017-3761eb64a75e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:46.665178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ecd2f-d165-4ba8-8339-f1c5a4a14073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:46.666431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ecd2f-d165-4ba8-8339-f1c5a4a14073 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:46.667581 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ecd2f-d165-4ba8-8339-f1c5a4a14073 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:46.668061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ecd2f-d165-4ba8-8339-f1c5a4a14073 None admin] PATH_INFO: `/v3/projects/1e503e3101124c66a9726948b602dc87` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:46.669662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ecd2f-d165-4ba8-8339-f1c5a4a14073 None admin] RBAC: Authorizing `identity:get_project(project_id=1e503e3101124c66a9726948b602dc87)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:46.671983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ecd2f-d165-4ba8-8339-f1c5a4a14073 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:46.674492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 195/412] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:46 2022] GET /identity/v3/projects/1e503e3101124c66a9726948b602dc87 => generated 344 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:46.685361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ecd2f-d165-4ba8-8339-f1c5a4a14073 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:46.691513 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05098e02-06a2-497e-a60c-e3d9c856b02c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:46.730259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320983e-8be5-4529-8015-c6685e4353cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:46.736706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320983e-8be5-4529-8015-c6685e4353cf None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:46.736706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320983e-8be5-4529-8015-c6685e4353cf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:46.736706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320983e-8be5-4529-8015-c6685e4353cf None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:46.736706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320983e-8be5-4529-8015-c6685e4353cf None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:46.736706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320983e-8be5-4529-8015-c6685e4353cf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:46.737351 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 218/413] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:46 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:46.748219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320983e-8be5-4529-8015-c6685e4353cf None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:46.807753 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:46.904493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320983e-8be5-4529-8015-c6685e4353cf None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:46.923751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:46.927951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ecd2f-d165-4ba8-8339-f1c5a4a14073 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:46.977844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb77597-be2d-4af9-9060-44892825769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:46.978737 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77597-be2d-4af9-9060-44892825769f None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:46.979025 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77597-be2d-4af9-9060-44892825769f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:46.979299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77597-be2d-4af9-9060-44892825769f None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793/groups/32a5071d1aef4422bbd2629d8bb75103/roles/2d0e0231af5d4e51b80de6bb65e8b02b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:46.984824 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb77597-be2d-4af9-9060-44892825769f None admin] RBAC: Authorizing `identity:create_grant(project_id=fcf4e9bfa6db4a5686e55857c6019793, group_id=32a5071d1aef4422bbd2629d8bb75103, role_id=2d0e0231af5d4e51b80de6bb65e8b02b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:46.986291 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb77597-be2d-4af9-9060-44892825769f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:46.995500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8000e-7ed5-41c5-a0de-6a820bcf6cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:46.996396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8000e-7ed5-41c5-a0de-6a820bcf6cce None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:46.996698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8000e-7ed5-41c5-a0de-6a820bcf6cce None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:46.997015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8000e-7ed5-41c5-a0de-6a820bcf6cce None admin] PATH_INFO: `/v3/projects/1e503e3101124c66a9726948b602dc87/groups/71eb78847cac4374b5b516fdfa03dcac/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:46.999162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8000e-7ed5-41c5-a0de-6a820bcf6cce None admin] RBAC: Authorizing `identity:create_grant(project_id=1e503e3101124c66a9726948b602dc87, group_id=71eb78847cac4374b5b516fdfa03dcac, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:47.000613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8000e-7ed5-41c5-a0de-6a820bcf6cce None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:47.010361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 219/414] 10.222.0.67 () {58 vars in 1442 bytes} [Thu Jun 23 11:41:46 2022] PUT /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793/groups/32a5071d1aef4422bbd2629d8bb75103/roles/2d0e0231af5d4e51b80de6bb65e8b02b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:41:47.011782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166a3b2-378b-4c89-b793-3802c8548171 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:47.012080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166a3b2-378b-4c89-b793-3802c8548171 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:47.012348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166a3b2-378b-4c89-b793-3802c8548171 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:47.018713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 196/416] 10.222.0.67 () {58 vars in 1442 bytes} [Thu Jun 23 11:41:46 2022] PUT /identity/v3/projects/1e503e3101124c66a9726948b602dc87/groups/71eb78847cac4374b5b516fdfa03dcac/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:41:47.018907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 220/416] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:47.020265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d05cd-65e8-4732-a84a-f63823c0601c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:47.020569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d05cd-65e8-4732-a84a-f63823c0601c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:47.020885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904d05cd-65e8-4732-a84a-f63823c0601c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:47.047067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-904d05cd-65e8-4732-a84a-f63823c0601c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:47.048884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-904d05cd-65e8-4732-a84a-f63823c0601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:47.136582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 221/417] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:47.173272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8000e-7ed5-41c5-a0de-6a820bcf6cce None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:47.339963 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8000e-7ed5-41c5-a0de-6a820bcf6cce None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:47.381137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9aa4e3-8965-4910-8be7-d15e7d71fc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:47.382446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9aa4e3-8965-4910-8be7-d15e7d71fc06 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:47.382986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9aa4e3-8965-4910-8be7-d15e7d71fc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:47.383474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9aa4e3-8965-4910-8be7-d15e7d71fc06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:47.402513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9aa4e3-8965-4910-8be7-d15e7d71fc06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-5c9aa4e3-8965-4910-8be7-d15e7d71fc06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:47.412205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:47.413283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:47.414268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:47.414268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:47.414268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:47.414268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:47.414268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:47.414268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:47.414268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:47.414268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:47.414268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:47.414268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:47.414268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:47.414268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:47.416970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 197/418] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:47.424762 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d05cd-65e8-4732-a84a-f63823c0601c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:47.472036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:47.531582 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac3cd6-c93f-4819-9b81-8dc94f922a65 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:47.532385 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac3cd6-c93f-4819-9b81-8dc94f922a65 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:47.532862 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac3cd6-c93f-4819-9b81-8dc94f922a65 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:47.534621 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 198/419] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:47.547319 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c025c-e97b-4a06-bd5b-9c38f079f15a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:47.547809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c025c-e97b-4a06-bd5b-9c38f079f15a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:47.548295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c025c-e97b-4a06-bd5b-9c38f079f15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:47.596720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-445c025c-e97b-4a06-bd5b-9c38f079f15a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:47.598625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-445c025c-e97b-4a06-bd5b-9c38f079f15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:47.607767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904d05cd-65e8-4732-a84a-f63823c0601c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:47.621128 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:47.696327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 199/420] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:47.727088 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7bcef-0bcf-49ca-b20b-74c6a9549c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:47.727791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7bcef-0bcf-49ca-b20b-74c6a9549c0c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:47.728084 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7bcef-0bcf-49ca-b20b-74c6a9549c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:47.728377 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7bcef-0bcf-49ca-b20b-74c6a9549c0c None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:47.728753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c025c-e97b-4a06-bd5b-9c38f079f15a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:47.729236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7bcef-0bcf-49ca-b20b-74c6a9549c0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:47.729552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7bcef-0bcf-49ca-b20b-74c6a9549c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:47.730955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7bcef-0bcf-49ca-b20b-74c6a9549c0c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:47.746839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 222/421] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:47.753311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7bcef-0bcf-49ca-b20b-74c6a9549c0c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:47.869221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:47.880195 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7bcef-0bcf-49ca-b20b-74c6a9549c0c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:47.905952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c025c-e97b-4a06-bd5b-9c38f079f15a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:47.922812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d30ae-8c57-4f4d-a4b0-247a8a0fee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:47.923876 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d30ae-8c57-4f4d-a4b0-247a8a0fee0d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:47.924516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d30ae-8c57-4f4d-a4b0-247a8a0fee0d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:47.925038 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9d30ae-8c57-4f4d-a4b0-247a8a0fee0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:48.013751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d30ae-8c57-4f4d-a4b0-247a8a0fee0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:48.015621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9d30ae-8c57-4f4d-a4b0-247a8a0fee0d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:48.038067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cb571-4731-476f-b351-6dd5fc20e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:48.038893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cb571-4731-476f-b351-6dd5fc20e44e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:48.039189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cb571-4731-476f-b351-6dd5fc20e44e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:48.039861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cb571-4731-476f-b351-6dd5fc20e44e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:48.085451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cb571-4731-476f-b351-6dd5fc20e44e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b58cb571-4731-476f-b351-6dd5fc20e44e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:48.093378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:48.094314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:48.095113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:48.095113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:48.095113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:48.095113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:48.095113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:48.095113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:48.095113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:48.095113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:48.095113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:48.095113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:48.095113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:48.095113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:48.109028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 200/422] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:41:48.113749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cb571-4731-476f-b351-6dd5fc20e44e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:48.129188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:48.143498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:48.151668 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 223/423] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 11:41:48.158100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d30ae-8c57-4f4d-a4b0-247a8a0fee0d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:48.299110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9d30ae-8c57-4f4d-a4b0-247a8a0fee0d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:48.367440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710c5cb-de35-45cc-8022-5ce4c7457c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:48.368712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710c5cb-de35-45cc-8022-5ce4c7457c8d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:48.369200 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710c5cb-de35-45cc-8022-5ce4c7457c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:48.369727 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710c5cb-de35-45cc-8022-5ce4c7457c8d None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:48.371360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710c5cb-de35-45cc-8022-5ce4c7457c8d None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:48.373092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710c5cb-de35-45cc-8022-5ce4c7457c8d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:48.375988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 224/424] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:48 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:48.381801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710c5cb-de35-45cc-8022-5ce4c7457c8d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:48.414736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cb571-4731-476f-b351-6dd5fc20e44e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:48.453275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:48.495720 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710c5cb-de35-45cc-8022-5ce4c7457c8d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:48.538667 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e2555-cf3f-45c7-a93b-6d89db94624e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:48.538667 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e2555-cf3f-45c7-a93b-6d89db94624e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:48.538667 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e2555-cf3f-45c7-a93b-6d89db94624e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:48.538667 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e2555-cf3f-45c7-a93b-6d89db94624e None admin] PATH_INFO: `/v3/groups/32a5071d1aef4422bbd2629d8bb75103` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:48.540242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e2555-cf3f-45c7-a93b-6d89db94624e None admin] RBAC: Authorizing `identity:get_group(group_id=32a5071d1aef4422bbd2629d8bb75103)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:48.542000 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845e2555-cf3f-45c7-a93b-6d89db94624e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:48.544233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 225/425] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:48 2022] GET /identity/v3/groups/32a5071d1aef4422bbd2629d8bb75103 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:48.545794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ff2ce-ab2a-4d6d-848d-1722f9d058b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:48.547062 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ff2ce-ab2a-4d6d-848d-1722f9d058b5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:48.547353 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ff2ce-ab2a-4d6d-848d-1722f9d058b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:48.547634 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ff2ce-ab2a-4d6d-848d-1722f9d058b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:48.548445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ff2ce-ab2a-4d6d-848d-1722f9d058b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:48.548786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ff2ce-ab2a-4d6d-848d-1722f9d058b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:48.550280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ff2ce-ab2a-4d6d-848d-1722f9d058b5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:48.552987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e2555-cf3f-45c7-a93b-6d89db94624e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:48.570954 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 201/426] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:48.574865 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ff2ce-ab2a-4d6d-848d-1722f9d058b5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:48.691451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:48.735895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ff2ce-ab2a-4d6d-848d-1722f9d058b5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:48.744665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e2555-cf3f-45c7-a93b-6d89db94624e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:48.778964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13e7b-ffa3-4624-89f7-410197cd1a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:48.779994 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13e7b-ffa3-4624-89f7-410197cd1a0a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:48.781021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13e7b-ffa3-4624-89f7-410197cd1a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:48.781482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d13e7b-ffa3-4624-89f7-410197cd1a0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:48.785146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:48.811409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360faec5-70bc-4e81-92cf-50ff9b0353ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:48.812472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360faec5-70bc-4e81-92cf-50ff9b0353ca None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:48.812872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360faec5-70bc-4e81-92cf-50ff9b0353ca None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:48.813223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360faec5-70bc-4e81-92cf-50ff9b0353ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:48.814274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360faec5-70bc-4e81-92cf-50ff9b0353ca None admin] RBAC: Adding query filter params (group.id=32a5071d1aef4422bbd2629d8bb75103, role.id=2d0e0231af5d4e51b80de6bb65e8b02b, scope.project.id=ea37fde41a0c43a790da11fb742b5490) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:48.814595 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360faec5-70bc-4e81-92cf-50ff9b0353ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:48.816271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360faec5-70bc-4e81-92cf-50ff9b0353ca None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:48.833079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 226/427] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 23 11:41:48 2022] GET /identity/v3/role_assignments?group.id=32a5071d1aef4422bbd2629d8bb75103&scope.project.id=ea37fde41a0c43a790da11fb742b5490&role.id=2d0e0231af5d4e51b80de6bb65e8b02b => generated 611 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:48.835897 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3703b7-6c3d-42d9-a7f5-048a30a55e6b None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:48.836501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3703b7-6c3d-42d9-a7f5-048a30a55e6b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:48.837122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3703b7-6c3d-42d9-a7f5-048a30a55e6b None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:48.840557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 227/428] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:48.853433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80dfa1b-b496-4b81-8642-1ed3e2303cfa None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:48.853695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80dfa1b-b496-4b81-8642-1ed3e2303cfa None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:48.854320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80dfa1b-b496-4b81-8642-1ed3e2303cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:48.878878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d13e7b-ffa3-4624-89f7-410197cd1a0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:48.881560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d13e7b-ffa3-4624-89f7-410197cd1a0a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:48.902585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e80dfa1b-b496-4b81-8642-1ed3e2303cfa None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:48.904847 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e80dfa1b-b496-4b81-8642-1ed3e2303cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:48.968864 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:49.014261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 202/429] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 11:41:49.016539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13e7b-ffa3-4624-89f7-410197cd1a0a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:49.110864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 228/430] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:49.166703 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80dfa1b-b496-4b81-8642-1ed3e2303cfa None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:49.190870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d13e7b-ffa3-4624-89f7-410197cd1a0a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:49.230957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732e20e-cd0f-46bc-845d-a3735346eeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:49.231759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732e20e-cd0f-46bc-845d-a3735346eeed None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:49.232053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732e20e-cd0f-46bc-845d-a3735346eeed None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:49.232335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732e20e-cd0f-46bc-845d-a3735346eeed None admin] PATH_INFO: `/v3/projects/ea37fde41a0c43a790da11fb742b5490` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:49.234262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732e20e-cd0f-46bc-845d-a3735346eeed None admin] RBAC: Authorizing `identity:get_project(project_id=ea37fde41a0c43a790da11fb742b5490)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:49.236019 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732e20e-cd0f-46bc-845d-a3735346eeed None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:49.242135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 203/431] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:49 2022] GET /identity/v3/projects/ea37fde41a0c43a790da11fb742b5490 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:49.244025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732e20e-cd0f-46bc-845d-a3735346eeed None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:49.295256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:49.303179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80dfa1b-b496-4b81-8642-1ed3e2303cfa None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:49.322715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:49.360933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2adb2-83e0-43dd-bc70-d61ff7e09f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:49.362319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2adb2-83e0-43dd-bc70-d61ff7e09f6a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:49.362623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2adb2-83e0-43dd-bc70-d61ff7e09f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:49.362911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2adb2-83e0-43dd-bc70-d61ff7e09f6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:49.380113 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2adb2-83e0-43dd-bc70-d61ff7e09f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:49.389727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732e20e-cd0f-46bc-845d-a3735346eeed None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-7da2adb2-83e0-43dd-bc70-d61ff7e09f6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:49.391167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:49.392274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:49.393627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 229/432] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:49.400533 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2adb2-83e0-43dd-bc70-d61ff7e09f6a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:49.446066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799fb6f-f510-4765-a686-dc0dc3f4bc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:49.448929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799fb6f-f510-4765-a686-dc0dc3f4bc6e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:49.449636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799fb6f-f510-4765-a686-dc0dc3f4bc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:49.450094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799fb6f-f510-4765-a686-dc0dc3f4bc6e None admin] PATH_INFO: `/v3/groups/32a5071d1aef4422bbd2629d8bb75103` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:49.452125 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799fb6f-f510-4765-a686-dc0dc3f4bc6e None admin] RBAC: Authorizing `identity:get_group(group_id=32a5071d1aef4422bbd2629d8bb75103)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:49.455035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799fb6f-f510-4765-a686-dc0dc3f4bc6e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:49.462402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 204/433] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:49 2022] GET /identity/v3/groups/32a5071d1aef4422bbd2629d8bb75103 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:49.466426 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799fb6f-f510-4765-a686-dc0dc3f4bc6e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:49.536863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2adb2-83e0-43dd-bc70-d61ff7e09f6a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:49.576641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc513256-3108-4901-a861-bdcf64cd969c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:49.577663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc513256-3108-4901-a861-bdcf64cd969c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:49.578107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc513256-3108-4901-a861-bdcf64cd969c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:49.578393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc513256-3108-4901-a861-bdcf64cd969c None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:49.579221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc513256-3108-4901-a861-bdcf64cd969c None admin] RBAC: Adding query filter params (name=member) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:49.579582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc513256-3108-4901-a861-bdcf64cd969c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:49.581100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc513256-3108-4901-a861-bdcf64cd969c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:49.581270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799fb6f-f510-4765-a686-dc0dc3f4bc6e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:49.589956 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:49.595586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 230/434] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:41:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:49.598111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905dd2be-e22f-4ada-acf8-1eba5cf37c26 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:49.598752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905dd2be-e22f-4ada-acf8-1eba5cf37c26 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:49.599364 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905dd2be-e22f-4ada-acf8-1eba5cf37c26 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:49.606399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 231/435] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:49.606399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905dd2be-e22f-4ada-acf8-1eba5cf37c26 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:49.633366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4afff3-9a8c-496f-9fe6-6a3a41bbd883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:49.634039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4afff3-9a8c-496f-9fe6-6a3a41bbd883 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:49.634636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4afff3-9a8c-496f-9fe6-6a3a41bbd883 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:49.634992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4afff3-9a8c-496f-9fe6-6a3a41bbd883 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:49.635757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4afff3-9a8c-496f-9fe6-6a3a41bbd883 None admin] RBAC: Adding query filter params (group.id=32a5071d1aef4422bbd2629d8bb75103, role.id=20703235b4c543bc9c51fb7c26da270d, scope.project.id=ea37fde41a0c43a790da11fb742b5490) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:49.636079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4afff3-9a8c-496f-9fe6-6a3a41bbd883 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:49.638329 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4afff3-9a8c-496f-9fe6-6a3a41bbd883 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:49.657278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 205/436] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 23 11:41:49 2022] GET /identity/v3/role_assignments?group.id=32a5071d1aef4422bbd2629d8bb75103&scope.project.id=ea37fde41a0c43a790da11fb742b5490&role.id=20703235b4c543bc9c51fb7c26da270d => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:49.657278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7ef59-d7d7-421d-9d21-5ceacf896fe3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:49.657278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7ef59-d7d7-421d-9d21-5ceacf896fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:49.657278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7ef59-d7d7-421d-9d21-5ceacf896fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:49.684187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:49.700518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bbe7ef59-d7d7-421d-9d21-5ceacf896fe3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:49.702882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bbe7ef59-d7d7-421d-9d21-5ceacf896fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:49.769108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905dd2be-e22f-4ada-acf8-1eba5cf37c26 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:49.803164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 206/437] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:49.814786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5991941-bc09-4a45-8db4-25b2e4c69ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:49.815858 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5991941-bc09-4a45-8db4-25b2e4c69ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:49.816332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5991941-bc09-4a45-8db4-25b2e4c69ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:49.816828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5991941-bc09-4a45-8db4-25b2e4c69ac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:49.828610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7ef59-d7d7-421d-9d21-5ceacf896fe3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:49.917323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5991941-bc09-4a45-8db4-25b2e4c69ac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:49.918822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5991941-bc09-4a45-8db4-25b2e4c69ac3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:49.964611 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:49.968923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:49.974182 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7ef59-d7d7-421d-9d21-5ceacf896fe3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:50.042844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456f708-562a-483b-92ae-a55e5dfa0b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:50.043832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f708-562a-483b-92ae-a55e5dfa0b05 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:50.044322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f708-562a-483b-92ae-a55e5dfa0b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:50.046353 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f708-562a-483b-92ae-a55e5dfa0b05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:50.065324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456f708-562a-483b-92ae-a55e5dfa0b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0456f708-562a-483b-92ae-a55e5dfa0b05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:50.073721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:50.075365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:50.076163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 207/438] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:50.077346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2880fe-eb54-403f-8991-f00084fcd065 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:50.077346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2880fe-eb54-403f-8991-f00084fcd065 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:50.077346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2880fe-eb54-403f-8991-f00084fcd065 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:50.079186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 208/439] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:50.082901 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2880fe-eb54-403f-8991-f00084fcd065 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:50.089101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 232/440] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:50.091623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54611c25-80f2-4b8b-ae88-e2bf38f2026f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:50.092026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54611c25-80f2-4b8b-ae88-e2bf38f2026f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:50.092413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54611c25-80f2-4b8b-ae88-e2bf38f2026f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:50.112974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-54611c25-80f2-4b8b-ae88-e2bf38f2026f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:50.114210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-54611c25-80f2-4b8b-ae88-e2bf38f2026f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:50.208571 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 233/441] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:50.213670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54611c25-80f2-4b8b-ae88-e2bf38f2026f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:50.214810 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2880fe-eb54-403f-8991-f00084fcd065 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:50.264071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111ce34-e1a4-4963-afed-b7ad05d5b6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:50.266555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111ce34-e1a4-4963-afed-b7ad05d5b6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:50.266555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111ce34-e1a4-4963-afed-b7ad05d5b6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:50.266555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a111ce34-e1a4-4963-afed-b7ad05d5b6e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:50.267809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111ce34-e1a4-4963-afed-b7ad05d5b6e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:50.269568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111ce34-e1a4-4963-afed-b7ad05d5b6e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:50.273456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a111ce34-e1a4-4963-afed-b7ad05d5b6e4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:50.285166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 209/442] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:50.285872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111ce34-e1a4-4963-afed-b7ad05d5b6e4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:50.303915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:50.317902 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:50.350360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54611c25-80f2-4b8b-ae88-e2bf38f2026f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:50.398613 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07975e6e-5cd3-4f40-bc69-14b2f04b06d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:50.399485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07975e6e-5cd3-4f40-bc69-14b2f04b06d4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:50.399770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07975e6e-5cd3-4f40-bc69-14b2f04b06d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:50.400050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07975e6e-5cd3-4f40-bc69-14b2f04b06d4 None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:50.401347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07975e6e-5cd3-4f40-bc69-14b2f04b06d4 None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:50.402817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07975e6e-5cd3-4f40-bc69-14b2f04b06d4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:50.404769 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 234/443] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:50 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:50.406041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07975e6e-5cd3-4f40-bc69-14b2f04b06d4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:50.434898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a111ce34-e1a4-4963-afed-b7ad05d5b6e4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:50.477350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3669e6-b0c8-476c-880d-700274dce383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:50.478288 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3669e6-b0c8-476c-880d-700274dce383 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:50.478761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3669e6-b0c8-476c-880d-700274dce383 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:50.479057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3669e6-b0c8-476c-880d-700274dce383 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:50.491216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3669e6-b0c8-476c-880d-700274dce383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0e3669e6-b0c8-476c-880d-700274dce383 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:50.499643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:50.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 210/444] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:41:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:50.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3669e6-b0c8-476c-880d-700274dce383 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:50.529594 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07975e6e-5cd3-4f40-bc69-14b2f04b06d4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:50.569846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856ea76-3447-414d-90db-c3004597074c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:50.570506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856ea76-3447-414d-90db-c3004597074c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:50.570889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856ea76-3447-414d-90db-c3004597074c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:50.571253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b856ea76-3447-414d-90db-c3004597074c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:50.574294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:50.600748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:50.620307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3669e6-b0c8-476c-880d-700274dce383 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:50.659419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c9774-8a7f-4f38-b297-a59e356806f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:50.659419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c9774-8a7f-4f38-b297-a59e356806f8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:50.659419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c9774-8a7f-4f38-b297-a59e356806f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:50.659419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c9774-8a7f-4f38-b297-a59e356806f8 None admin] PATH_INFO: `/v3/groups/32a5071d1aef4422bbd2629d8bb75103` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:50.662037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c9774-8a7f-4f38-b297-a59e356806f8 None admin] RBAC: Authorizing `identity:get_group(group_id=32a5071d1aef4422bbd2629d8bb75103)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:50.664533 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c9774-8a7f-4f38-b297-a59e356806f8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:50.667850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 211/445] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:50 2022] GET /identity/v3/groups/32a5071d1aef4422bbd2629d8bb75103 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:50.669257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c9774-8a7f-4f38-b297-a59e356806f8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:50.688247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856ea76-3447-414d-90db-c3004597074c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:50.691128 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b856ea76-3447-414d-90db-c3004597074c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:50.813981 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 235/446] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:50.815692 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856ea76-3447-414d-90db-c3004597074c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:50.818243 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c9774-8a7f-4f38-b297-a59e356806f8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:50.853149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01657d6-9f9d-473b-b700-4d03c5670fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:50.853743 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01657d6-9f9d-473b-b700-4d03c5670fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:50.854032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01657d6-9f9d-473b-b700-4d03c5670fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:50.854321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01657d6-9f9d-473b-b700-4d03c5670fd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:50.855083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01657d6-9f9d-473b-b700-4d03c5670fd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:50.855397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01657d6-9f9d-473b-b700-4d03c5670fd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:50.856803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01657d6-9f9d-473b-b700-4d03c5670fd0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:50.864043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:50.871701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 212/447] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:41:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:50.872617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01657d6-9f9d-473b-b700-4d03c5670fd0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:50.887996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:50.941342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b856ea76-3447-414d-90db-c3004597074c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:50.977357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625cf7c-990d-4b2b-9866-aca74f2022e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:50.978017 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625cf7c-990d-4b2b-9866-aca74f2022e6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:50.978327 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625cf7c-990d-4b2b-9866-aca74f2022e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:50.978614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625cf7c-990d-4b2b-9866-aca74f2022e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:50.979503 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625cf7c-990d-4b2b-9866-aca74f2022e6 None admin] RBAC: Adding query filter params (group.id=32a5071d1aef4422bbd2629d8bb75103, role.id=20703235b4c543bc9c51fb7c26da270d, scope.project.id=fcf4e9bfa6db4a5686e55857c6019793) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:50.979824 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625cf7c-990d-4b2b-9866-aca74f2022e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:50.980000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01657d6-9f9d-473b-b700-4d03c5670fd0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:50.981301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625cf7c-990d-4b2b-9866-aca74f2022e6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:50.988198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 236/448] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 23 11:41:50 2022] GET /identity/v3/role_assignments?group.id=32a5071d1aef4422bbd2629d8bb75103&scope.project.id=fcf4e9bfa6db4a5686e55857c6019793&role.id=20703235b4c543bc9c51fb7c26da270d => generated 611 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:50.989240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625cf7c-990d-4b2b-9866-aca74f2022e6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:51.032720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682d50f-b48d-467a-aac5-89c151985fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:51.034010 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682d50f-b48d-467a-aac5-89c151985fbe None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:51.034301 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682d50f-b48d-467a-aac5-89c151985fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:51.034584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4682d50f-b48d-467a-aac5-89c151985fbe None admin] PATH_INFO: `/v3/projects/fcf4e9bfa6db4a5686e55857c6019793` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:51.036124 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682d50f-b48d-467a-aac5-89c151985fbe None admin] RBAC: Authorizing `identity:get_project(project_id=fcf4e9bfa6db4a5686e55857c6019793)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:51.037683 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4682d50f-b48d-467a-aac5-89c151985fbe None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:51.039740 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 213/449] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:50 2022] GET /identity/v3/projects/fcf4e9bfa6db4a5686e55857c6019793 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:51.046458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682d50f-b48d-467a-aac5-89c151985fbe None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:51.117674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:51.122511 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625cf7c-990d-4b2b-9866-aca74f2022e6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:51.162847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:51.179314 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb8d2a-f2b3-4cb0-af8d-1695f09a2a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:51.179977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb8d2a-f2b3-4cb0-af8d-1695f09a2a90 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:51.180260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb8d2a-f2b3-4cb0-af8d-1695f09a2a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:51.180700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb8d2a-f2b3-4cb0-af8d-1695f09a2a90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:51.190216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4682d50f-b48d-467a-aac5-89c151985fbe None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:51.233538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffebe1-dca5-49ed-9c8a-cb61cb82f971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:51.234504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffebe1-dca5-49ed-9c8a-cb61cb82f971 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:51.234894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffebe1-dca5-49ed-9c8a-cb61cb82f971 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:51.235287 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffebe1-dca5-49ed-9c8a-cb61cb82f971 None admin] PATH_INFO: `/v3/groups/32a5071d1aef4422bbd2629d8bb75103` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:51.236798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffebe1-dca5-49ed-9c8a-cb61cb82f971 None admin] RBAC: Authorizing `identity:get_group(group_id=32a5071d1aef4422bbd2629d8bb75103)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:51.238494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffebe1-dca5-49ed-9c8a-cb61cb82f971 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:51.244751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 214/450] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:51 2022] GET /identity/v3/groups/32a5071d1aef4422bbd2629d8bb75103 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:51.246700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffebe1-dca5-49ed-9c8a-cb61cb82f971 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:51.281339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb8d2a-f2b3-4cb0-af8d-1695f09a2a90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:51.283147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cb8d2a-f2b3-4cb0-af8d-1695f09a2a90 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:51.390230 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffebe1-dca5-49ed-9c8a-cb61cb82f971 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:51.404050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:51.410424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 237/451] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:41:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:41:51.416141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb8d2a-f2b3-4cb0-af8d-1695f09a2a90 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:51.432519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296cd85a-87f1-4816-942f-6049b7d977b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:51.433421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cd85a-87f1-4816-942f-6049b7d977b8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:51.433795 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cd85a-87f1-4816-942f-6049b7d977b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:51.434207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cd85a-87f1-4816-942f-6049b7d977b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:51.435148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296cd85a-87f1-4816-942f-6049b7d977b8 None admin] RBAC: Adding query filter params (group.id=32a5071d1aef4422bbd2629d8bb75103, role.id=2d0e0231af5d4e51b80de6bb65e8b02b, scope.project.id=fcf4e9bfa6db4a5686e55857c6019793) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:51.435559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296cd85a-87f1-4816-942f-6049b7d977b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:51.437201 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296cd85a-87f1-4816-942f-6049b7d977b8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:51.443972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 215/452] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 23 11:41:51 2022] GET /identity/v3/role_assignments?group.id=32a5071d1aef4422bbd2629d8bb75103&scope.project.id=fcf4e9bfa6db4a5686e55857c6019793&role.id=2d0e0231af5d4e51b80de6bb65e8b02b => generated 611 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:51.530088 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb8d2a-f2b3-4cb0-af8d-1695f09a2a90 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:51.569158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26981f2a-2155-4dc2-8261-368074f7b1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:51.570036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26981f2a-2155-4dc2-8261-368074f7b1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:51.570666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26981f2a-2155-4dc2-8261-368074f7b1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:51.570666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26981f2a-2155-4dc2-8261-368074f7b1f6 None admin] PATH_INFO: `/v3/projects/1e503e3101124c66a9726948b602dc87` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:51.571912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26981f2a-2155-4dc2-8261-368074f7b1f6 None admin] RBAC: Authorizing `identity:get_project(project_id=1e503e3101124c66a9726948b602dc87)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:51.573433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26981f2a-2155-4dc2-8261-368074f7b1f6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:51.575224 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 238/453] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:41:51 2022] GET /identity/v3/projects/1e503e3101124c66a9726948b602dc87 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:51.581650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296cd85a-87f1-4816-942f-6049b7d977b8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:51.598901 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:51.711523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296cd85a-87f1-4816-942f-6049b7d977b8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:51.750561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8995d5f-0584-4a7d-bd92-171be48190fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:51.751421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8995d5f-0584-4a7d-bd92-171be48190fc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:51.751701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8995d5f-0584-4a7d-bd92-171be48190fc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:51.751976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8995d5f-0584-4a7d-bd92-171be48190fc None admin] PATH_INFO: `/v3/groups/71eb78847cac4374b5b516fdfa03dcac` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:51.753305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8995d5f-0584-4a7d-bd92-171be48190fc None admin] RBAC: Authorizing `identity:get_group(group_id=71eb78847cac4374b5b516fdfa03dcac)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:51.754762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8995d5f-0584-4a7d-bd92-171be48190fc None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:51.756438 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 216/454] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 23 11:41:51 2022] GET /identity/v3/groups/71eb78847cac4374b5b516fdfa03dcac => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:51.762787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26981f2a-2155-4dc2-8261-368074f7b1f6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:51.840980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:51.877025 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26981f2a-2155-4dc2-8261-368074f7b1f6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:51.911450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237f1d4-222a-4479-b7ba-7a9c51e746aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:51.912100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237f1d4-222a-4479-b7ba-7a9c51e746aa None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:51.912388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237f1d4-222a-4479-b7ba-7a9c51e746aa None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:51.912703 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237f1d4-222a-4479-b7ba-7a9c51e746aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:51.913535 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237f1d4-222a-4479-b7ba-7a9c51e746aa None admin] RBAC: Adding query filter params (group.id=71eb78847cac4374b5b516fdfa03dcac, role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=1e503e3101124c66a9726948b602dc87) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:51.913852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237f1d4-222a-4479-b7ba-7a9c51e746aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:51.915297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237f1d4-222a-4479-b7ba-7a9c51e746aa None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:51.921676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 239/455] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 23 11:41:51 2022] GET /identity/v3/role_assignments?group.id=71eb78847cac4374b5b516fdfa03dcac&scope.project.id=1e503e3101124c66a9726948b602dc87&role.id=569707378794437b93d168fc8bb9cbc3 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:54.503008 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea55b6b-5450-487c-b862-33f2c7ea2a2e None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:54.503538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea55b6b-5450-487c-b862-33f2c7ea2a2e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:54.503643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea55b6b-5450-487c-b862-33f2c7ea2a2e None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:54.505640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 217/456] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:54.512432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bd765-e397-4b42-a7a6-d602771fd844 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:54.513464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bd765-e397-4b42-a7a6-d602771fd844 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:54.513968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bd765-e397-4b42-a7a6-d602771fd844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:54.535105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-747bd765-e397-4b42-a7a6-d602771fd844 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:54.536954 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-747bd765-e397-4b42-a7a6-d602771fd844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:54.615186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 240/457] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:54.654914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea55b6b-5450-487c-b862-33f2c7ea2a2e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:54.771651 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831ab23-3b84-424b-b895-2a11e531c6d3 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:54.772284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831ab23-3b84-424b-b895-2a11e531c6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:54.773108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831ab23-3b84-424b-b895-2a11e531c6d3 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:54.776807 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 241/458] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:54.784931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f28f1c-df3b-4991-aca0-dab65edbd8cf None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:54.785582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f28f1c-df3b-4991-aca0-dab65edbd8cf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:54.786065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f28f1c-df3b-4991-aca0-dab65edbd8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:54.816705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-28f28f1c-df3b-4991-aca0-dab65edbd8cf None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:54.818646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-28f28f1c-df3b-4991-aca0-dab65edbd8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:54.824219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:54.848631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea55b6b-5450-487c-b862-33f2c7ea2a2e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:54.889138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8f9a2c-8f2e-47f4-859f-bec763d93e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:54.890646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f9a2c-8f2e-47f4-859f-bec763d93e76 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:54.890929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f9a2c-8f2e-47f4-859f-bec763d93e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:54.891210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8f9a2c-8f2e-47f4-859f-bec763d93e76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:54.903709 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f9a2c-8f2e-47f4-859f-bec763d93e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:54.905273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8f9a2c-8f2e-47f4-859f-bec763d93e76 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:54.918083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-8a8f9a2c-8f2e-47f4-859f-bec763d93e76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:41:54.927918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:54.929557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:54.931242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:41:54.931242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:41:54.931242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:41:54.931242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:54.931242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 218/459] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:41:54.931242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96813003-44b7-47c9-985e-c44c671fa6a4 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:54.931242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96813003-44b7-47c9-985e-c44c671fa6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:54.931983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96813003-44b7-47c9-985e-c44c671fa6a4 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:54.932950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 219/460] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:54.942571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96813003-44b7-47c9-985e-c44c671fa6a4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:54.960253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 242/461] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:54.963176 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71dd46-ee42-480b-a62c-39848bc333ff None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:54.963909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71dd46-ee42-480b-a62c-39848bc333ff None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:54.964455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71dd46-ee42-480b-a62c-39848bc333ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:54.997786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0f71dd46-ee42-480b-a62c-39848bc333ff None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:55.001242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0f71dd46-ee42-480b-a62c-39848bc333ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:55.076555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96813003-44b7-47c9-985e-c44c671fa6a4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:55.114899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:55.125832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9c90e-4c43-4192-964f-0a5c0dc4cdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:55.127305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9c90e-4c43-4192-964f-0a5c0dc4cdcb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:55.127857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9c90e-4c43-4192-964f-0a5c0dc4cdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:55.128797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9c90e-4c43-4192-964f-0a5c0dc4cdcb None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:55.130212 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9c90e-4c43-4192-964f-0a5c0dc4cdcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:55.131236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9c90e-4c43-4192-964f-0a5c0dc4cdcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:55.133397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9c90e-4c43-4192-964f-0a5c0dc4cdcb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:55.147860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 243/462] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:55.149682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71dd46-ee42-480b-a62c-39848bc333ff None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:55.159188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 220/463] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:55.186184 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9c90e-4c43-4192-964f-0a5c0dc4cdcb None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:55.343839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71dd46-ee42-480b-a62c-39848bc333ff None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:55.352067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9c90e-4c43-4192-964f-0a5c0dc4cdcb None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:55.385162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:55.415525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969fc5e-ac20-4133-bec2-0c846b8180cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:55.417390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969fc5e-ac20-4133-bec2-0c846b8180cc None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:55.424965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969fc5e-ac20-4133-bec2-0c846b8180cc None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:55.425916 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969fc5e-ac20-4133-bec2-0c846b8180cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:55.429495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df0721-0221-4718-b38b-083b6e3aafda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:55.430943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df0721-0221-4718-b38b-083b6e3aafda None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:55.431420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df0721-0221-4718-b38b-083b6e3aafda None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:55.431905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df0721-0221-4718-b38b-083b6e3aafda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:55.446934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969fc5e-ac20-4133-bec2-0c846b8180cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:55.453030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df0721-0221-4718-b38b-083b6e3aafda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:55.454517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969fc5e-ac20-4133-bec2-0c846b8180cc None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:55.455482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df0721-0221-4718-b38b-083b6e3aafda None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-c3df0721-0221-4718-b38b-083b6e3aafda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:41:55.480661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:55.481704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:55.482549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:41:55.482549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:41:55.482549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:41:55.482549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2969fc5e-ac20-4133-bec2-0c846b8180cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:41:55.483932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:55.485080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:55.485935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:41:55.485935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:41:55.485935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:41:55.485935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:55.487822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 244/464] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:55.489200 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969fc5e-ac20-4133-bec2-0c846b8180cc None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:55.489436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 221/465] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:55.494919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df0721-0221-4718-b38b-083b6e3aafda None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:55.575725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:55.650530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969fc5e-ac20-4133-bec2-0c846b8180cc None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:55.651234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df0721-0221-4718-b38b-083b6e3aafda None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:55.702031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27591515-e49e-4da4-aeb6-bde3ddb5674b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:55.703674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27591515-e49e-4da4-aeb6-bde3ddb5674b None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:55.704182 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27591515-e49e-4da4-aeb6-bde3ddb5674b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:55.704549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27591515-e49e-4da4-aeb6-bde3ddb5674b None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:55.705347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27591515-e49e-4da4-aeb6-bde3ddb5674b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:55.706913 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27591515-e49e-4da4-aeb6-bde3ddb5674b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:55.718305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665c372-8cd0-41ba-8e98-3d8574411c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:55.719566 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665c372-8cd0-41ba-8e98-3d8574411c73 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:55.719938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665c372-8cd0-41ba-8e98-3d8574411c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:55.720389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665c372-8cd0-41ba-8e98-3d8574411c73 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:55.721273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665c372-8cd0-41ba-8e98-3d8574411c73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:55.721615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665c372-8cd0-41ba-8e98-3d8574411c73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:55.723081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665c372-8cd0-41ba-8e98-3d8574411c73 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:55.741299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 222/466] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:55.741299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665c372-8cd0-41ba-8e98-3d8574411c73 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:55.751254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 245/467] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:41:55 2022] POST /identity/v3/users => generated 246 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:41:55.754382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805cc1a-41ad-46d4-aeab-0d2ce2a90fdd None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:55.754382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805cc1a-41ad-46d4-aeab-0d2ce2a90fdd None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:55.755239 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805cc1a-41ad-46d4-aeab-0d2ce2a90fdd None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:55.757993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 246/468] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:55.758702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805cc1a-41ad-46d4-aeab-0d2ce2a90fdd None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:55.826386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:55.950061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805cc1a-41ad-46d4-aeab-0d2ce2a90fdd None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:55.959108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665c372-8cd0-41ba-8e98-3d8574411c73 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:55.964468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:56.005946 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d359d9f-5b4d-47c9-9e5f-f1d8716ec711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:56.006809 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d359d9f-5b4d-47c9-9e5f-f1d8716ec711 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:56.007094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d359d9f-5b4d-47c9-9e5f-f1d8716ec711 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:56.007369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d359d9f-5b4d-47c9-9e5f-f1d8716ec711 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:56.008103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d359d9f-5b4d-47c9-9e5f-f1d8716ec711 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:56.009652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d359d9f-5b4d-47c9-9e5f-f1d8716ec711 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:56.026345 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd06ee-9992-4dca-b530-7c405d8a3583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:56.027054 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd06ee-9992-4dca-b530-7c405d8a3583 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:56.027327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd06ee-9992-4dca-b530-7c405d8a3583 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:56.027882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd06ee-9992-4dca-b530-7c405d8a3583 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:56.034702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd06ee-9992-4dca-b530-7c405d8a3583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:56.034702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd06ee-9992-4dca-b530-7c405d8a3583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:56.034702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd06ee-9992-4dca-b530-7c405d8a3583 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:56.050939 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 247/469] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:41:55 2022] POST /identity/v3/users => generated 248 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:41:56.054804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 223/470] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:56.057462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0fed8-ea5e-4f37-920e-fc12e11e1f3d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:56.058416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0fed8-ea5e-4f37-920e-fc12e11e1f3d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:56.059267 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0fed8-ea5e-4f37-920e-fc12e11e1f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:56.060426 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd06ee-9992-4dca-b530-7c405d8a3583 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:56.085314 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-aaf0fed8-ea5e-4f37-920e-fc12e11e1f3d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:56.087530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-aaf0fed8-ea5e-4f37-920e-fc12e11e1f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:56.208581 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd06ee-9992-4dca-b530-7c405d8a3583 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:56.226187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 248/471] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:56.251189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af379b8-bf7f-40f5-b7ac-959e29daf930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:56.251880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af379b8-bf7f-40f5-b7ac-959e29daf930 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:56.252161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af379b8-bf7f-40f5-b7ac-959e29daf930 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:56.252436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af379b8-bf7f-40f5-b7ac-959e29daf930 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:56.253337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af379b8-bf7f-40f5-b7ac-959e29daf930 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:56.254827 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af379b8-bf7f-40f5-b7ac-959e29daf930 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:56.255725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070d4dc-c378-4a59-8ffb-27444f97ac08 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:56.256072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070d4dc-c378-4a59-8ffb-27444f97ac08 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:56.256545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070d4dc-c378-4a59-8ffb-27444f97ac08 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:56.258434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 249/472] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:56.267654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4bcb65-512a-493a-b965-9842f0b74865 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:56.268052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4bcb65-512a-493a-b965-9842f0b74865 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:56.268543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4bcb65-512a-493a-b965-9842f0b74865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:56.281251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 224/473] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:41:56 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:41:56.282698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af379b8-bf7f-40f5-b7ac-959e29daf930 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:56.302654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ea4bcb65-512a-493a-b965-9842f0b74865 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:56.304761 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ea4bcb65-512a-493a-b965-9842f0b74865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:56.322830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:56.339425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:56.439857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 250/474] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:56.473174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4bcb65-512a-493a-b965-9842f0b74865 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:56.486741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af379b8-bf7f-40f5-b7ac-959e29daf930 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:56.525068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38074aa-ed10-4c34-8162-9f9d630c043b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:56.525875 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38074aa-ed10-4c34-8162-9f9d630c043b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:56.526165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38074aa-ed10-4c34-8162-9f9d630c043b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:56.526452 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38074aa-ed10-4c34-8162-9f9d630c043b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:56.539409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38074aa-ed10-4c34-8162-9f9d630c043b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:56.540961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38074aa-ed10-4c34-8162-9f9d630c043b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f38074aa-ed10-4c34-8162-9f9d630c043b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:41:56.564564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:56.565506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:56.567000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:41:56.567000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:41:56.567000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:41:56.567000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:56.567000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 225/475] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:56.574914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38074aa-ed10-4c34-8162-9f9d630c043b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:56.616659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4bcb65-512a-493a-b965-9842f0b74865 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:56.657402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3875c3-2040-4365-9257-bf525dda4fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:56.658159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3875c3-2040-4365-9257-bf525dda4fb7 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:56.658368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3875c3-2040-4365-9257-bf525dda4fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:56.658641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3875c3-2040-4365-9257-bf525dda4fb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:56.659434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3875c3-2040-4365-9257-bf525dda4fb7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:56.660901 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3875c3-2040-4365-9257-bf525dda4fb7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:56.673972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.sql.core [None req-1e3875c3-2040-4365-9257-bf525dda4fb7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 11:41:56.673972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 11:41:56.673972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [parameters: {'id': '3aa56a9b3f9f4639878fa0e609af26c5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 11:41:56.673972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80243) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-1e3875c3-2040-4365-9257-bf525dda4fb7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application conn.query(q) Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result.read() Jun 23 11:41:56.694219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return ref.to_dict() Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 11:41:56.695286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application next(self.gen) Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application yield resource Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application next(self.gen) Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise self.value Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application session.commit() Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 11:41:56.696402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.session.flush() Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application compat.raise_( Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 11:41:56.697424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 11:41:56.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application conn.query(q) Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result.read() Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 11:41:56.699352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [parameters: {'id': '3aa56a9b3f9f4639878fa0e609af26c5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:56.700265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 11:41:56.701500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:56.705011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 251/476] 10.222.0.67 () {60 vars in 1131 bytes} [Thu Jun 23 11:41:56 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:56.716585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:56.720164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3875c3-2040-4365-9257-bf525dda4fb7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:56.736225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:56.779934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38074aa-ed10-4c34-8162-9f9d630c043b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:56.870533 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6ff84-6843-48d1-9c7e-dc800b667e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:56.871514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6ff84-6843-48d1-9c7e-dc800b667e60 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:56.871954 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6ff84-6843-48d1-9c7e-dc800b667e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:56.872381 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6ff84-6843-48d1-9c7e-dc800b667e60 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:56.873334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6ff84-6843-48d1-9c7e-dc800b667e60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:56.873805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6ff84-6843-48d1-9c7e-dc800b667e60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:56.876085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6ff84-6843-48d1-9c7e-dc800b667e60 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:56.920289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3875c3-2040-4365-9257-bf525dda4fb7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:56.927484 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 226/477] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:56.953533 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6ff84-6843-48d1-9c7e-dc800b667e60 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:57.002940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171a517-8ab5-4944-ad22-e027d439682c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:57.003696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171a517-8ab5-4944-ad22-e027d439682c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:57.003986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171a517-8ab5-4944-ad22-e027d439682c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:57.004266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171a517-8ab5-4944-ad22-e027d439682c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:57.027052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171a517-8ab5-4944-ad22-e027d439682c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4171a517-8ab5-4944-ad22-e027d439682c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:57.035962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:57.036976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:57.037790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:57.037790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:57.037790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:57.037790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:57.037790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:57.037790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:57.037790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:57.037790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:57.037790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:57.037790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:57.037790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 11:41:57.037790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:57.041858 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 252/478] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:41:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:57.050663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171a517-8ab5-4944-ad22-e027d439682c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:57.189519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6ff84-6843-48d1-9c7e-dc800b667e60 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:57.242836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:57.248254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63433e4c-04ff-48ea-be59-bc7dfca99749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:57.249562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63433e4c-04ff-48ea-be59-bc7dfca99749 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:57.249844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63433e4c-04ff-48ea-be59-bc7dfca99749 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:57.250125 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63433e4c-04ff-48ea-be59-bc7dfca99749 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:57.250863 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63433e4c-04ff-48ea-be59-bc7dfca99749 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:57.252453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63433e4c-04ff-48ea-be59-bc7dfca99749 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:57.261664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:57.282445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 227/479] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:41:56 2022] POST /identity/v3/users => generated 249 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:41:57.323951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171a517-8ab5-4944-ad22-e027d439682c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:57.374673 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20641d8e-b969-4a79-8ef6-29dc20f460a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:57.375474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20641d8e-b969-4a79-8ef6-29dc20f460a6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:57.375819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20641d8e-b969-4a79-8ef6-29dc20f460a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:57.376100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20641d8e-b969-4a79-8ef6-29dc20f460a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:57.377000 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20641d8e-b969-4a79-8ef6-29dc20f460a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:57.377319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20641d8e-b969-4a79-8ef6-29dc20f460a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:57.378795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20641d8e-b969-4a79-8ef6-29dc20f460a6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:57.392267 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 253/480] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:41:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:58.591413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91107d3-2004-4614-9c00-fbcea8e981dd None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:58.591979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91107d3-2004-4614-9c00-fbcea8e981dd None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:58.592263 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91107d3-2004-4614-9c00-fbcea8e981dd None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:58.594660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 228/481] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:58.600569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2d6d2-0328-4b8f-9953-16a522c8e3cf None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:58.601123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2d6d2-0328-4b8f-9953-16a522c8e3cf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:58.601326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2d6d2-0328-4b8f-9953-16a522c8e3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:58.630001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-18c2d6d2-0328-4b8f-9953-16a522c8e3cf None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:58.632068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-18c2d6d2-0328-4b8f-9953-16a522c8e3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:58.787047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 254/482] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:58.826972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91107d3-2004-4614-9c00-fbcea8e981dd None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:58.867109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c6c7e-160c-47c9-b7de-271efd76755c None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:58.867842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c6c7e-160c-47c9-b7de-271efd76755c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:58.868330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c6c7e-160c-47c9-b7de-271efd76755c None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:58.870724 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 255/483] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:58.889770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631b323-cf2a-455f-846a-a32408d095bc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:58.890581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631b323-cf2a-455f-846a-a32408d095bc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:58.891522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631b323-cf2a-455f-846a-a32408d095bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:58.949138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d631b323-cf2a-455f-846a-a32408d095bc None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:58.952165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d631b323-cf2a-455f-846a-a32408d095bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:59.087666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91107d3-2004-4614-9c00-fbcea8e981dd None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:59.097271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:59.115838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 256/484] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:41:59.164324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51196462-5248-4adf-b6c7-43333ce599ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:59.165225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51196462-5248-4adf-b6c7-43333ce599ac None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:59.165511 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51196462-5248-4adf-b6c7-43333ce599ac None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:59.165787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51196462-5248-4adf-b6c7-43333ce599ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:59.178707 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51196462-5248-4adf-b6c7-43333ce599ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-51196462-5248-4adf-b6c7-43333ce599ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:59.186842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:59.187804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:59.188610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:59.188610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:59.188610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:59.188610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:59.188610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:59.188610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:59.188610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:59.188610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:59.188610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:59.188610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:59.188610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:41:59.188610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:59.192824 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 229/485] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:41:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:59.194521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631b323-cf2a-455f-846a-a32408d095bc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:59.204769 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51196462-5248-4adf-b6c7-43333ce599ac None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:59.274450 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:59.349786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51196462-5248-4adf-b6c7-43333ce599ac None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:59.383774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631b323-cf2a-455f-846a-a32408d095bc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:59.396663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae999f05-c1ac-49cc-8498-1efad148d185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:59.397696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae999f05-c1ac-49cc-8498-1efad148d185 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:59.398082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae999f05-c1ac-49cc-8498-1efad148d185 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:59.398465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae999f05-c1ac-49cc-8498-1efad148d185 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:59.413685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae999f05-c1ac-49cc-8498-1efad148d185 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-ae999f05-c1ac-49cc-8498-1efad148d185 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:59.426545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:59.427507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:59.428314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:59.428314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:59.428314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:59.428314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:59.428314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:59.428314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:59.428314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:59.428314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:59.428314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:59.428314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:59.428314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:41:59.428314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:59.434649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e98bec-65df-48ed-b289-2309417aa073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:59.435450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e98bec-65df-48ed-b289-2309417aa073 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:59.435558 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 230/486] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:41:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:59.435730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e98bec-65df-48ed-b289-2309417aa073 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:59.436014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e98bec-65df-48ed-b289-2309417aa073 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:59.436880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e98bec-65df-48ed-b289-2309417aa073 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:59.437196 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e98bec-65df-48ed-b289-2309417aa073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:59.437728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84c91e-6e01-4705-b5b6-87cce000c034 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:59.438137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84c91e-6e01-4705-b5b6-87cce000c034 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:59.438590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e84c91e-6e01-4705-b5b6-87cce000c034 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:59.438691 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e98bec-65df-48ed-b289-2309417aa073 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:59.441616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 231/487] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:41:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:41:59.442728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84c91e-6e01-4705-b5b6-87cce000c034 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:59.455542 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 257/488] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:41:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:41:59.457333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a85cb-cca1-4811-8c5b-317228224efc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:59.457770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a85cb-cca1-4811-8c5b-317228224efc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:59.458057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a85cb-cca1-4811-8c5b-317228224efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:59.480593 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-843a85cb-cca1-4811-8c5b-317228224efc None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:41:59.481904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-843a85cb-cca1-4811-8c5b-317228224efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:41:59.549342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:59.575550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 258/489] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:41:59.576659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a85cb-cca1-4811-8c5b-317228224efc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:59.610593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e84c91e-6e01-4705-b5b6-87cce000c034 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:59.630559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:59.676459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87823de-b2c3-43d2-aa66-1b8d8f434a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:59.677584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87823de-b2c3-43d2-aa66-1b8d8f434a2f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:59.677861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87823de-b2c3-43d2-aa66-1b8d8f434a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:59.678212 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87823de-b2c3-43d2-aa66-1b8d8f434a2f None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:59.680116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87823de-b2c3-43d2-aa66-1b8d8f434a2f None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:41:59.680508 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87823de-b2c3-43d2-aa66-1b8d8f434a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:59.682034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87823de-b2c3-43d2-aa66-1b8d8f434a2f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:59.694528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 232/490] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:41:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:41:59.700285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87823de-b2c3-43d2-aa66-1b8d8f434a2f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:59.725823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a85cb-cca1-4811-8c5b-317228224efc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:59.788491 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d48919-1922-4725-9538-bbc80290fd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:59.789700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d48919-1922-4725-9538-bbc80290fd65 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:59.790275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d48919-1922-4725-9538-bbc80290fd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:59.790845 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d48919-1922-4725-9538-bbc80290fd65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:59.804885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d48919-1922-4725-9538-bbc80290fd65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:41:59.807216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d48919-1922-4725-9538-bbc80290fd65 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:41:59.819770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87823de-b2c3-43d2-aa66-1b8d8f434a2f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-23d48919-1922-4725-9538-bbc80290fd65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:41:59.830371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:59.831286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:59.832178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:41:59.832178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:41:59.832178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:41:59.832178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:41:59.834958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 259/491] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:59.836134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d48919-1922-4725-9538-bbc80290fd65 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:59.862566 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231535cf-072f-4a01-aef1-ad4c03d312b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:41:59.863449 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231535cf-072f-4a01-aef1-ad4c03d312b2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:41:59.863779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231535cf-072f-4a01-aef1-ad4c03d312b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:41:59.864232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231535cf-072f-4a01-aef1-ad4c03d312b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:41:59.877548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231535cf-072f-4a01-aef1-ad4c03d312b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:41:59.881323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-231535cf-072f-4a01-aef1-ad4c03d312b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:41:59.886916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:41:59.888172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:41:59.889297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:41:59.889297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:41:59.889297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:41:59.889297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:41:59.889297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:41:59.889297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:41:59.889297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:41:59.889297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:41:59.889297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:41:59.889297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:41:59.889297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:41:59.889297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:41:59.892912 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 233/492] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:41:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:41:59.894061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231535cf-072f-4a01-aef1-ad4c03d312b2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:41:59.930798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:41:59.992152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d48919-1922-4725-9538-bbc80290fd65 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:00.018935 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231535cf-072f-4a01-aef1-ad4c03d312b2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:00.054187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c96738-3397-4f98-a6c4-8f6b1038469e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:00.055163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c96738-3397-4f98-a6c4-8f6b1038469e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.055577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c96738-3397-4f98-a6c4-8f6b1038469e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.055919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c96738-3397-4f98-a6c4-8f6b1038469e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.068655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c96738-3397-4f98-a6c4-8f6b1038469e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:00.070356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c96738-3397-4f98-a6c4-8f6b1038469e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:00.074453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad3e1f-a811-4423-9e66-7359351cbf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:00.075877 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad3e1f-a811-4423-9e66-7359351cbf12 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.076550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad3e1f-a811-4423-9e66-7359351cbf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.077845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad3e1f-a811-4423-9e66-7359351cbf12 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.079617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad3e1f-a811-4423-9e66-7359351cbf12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:00.080485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad3e1f-a811-4423-9e66-7359351cbf12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:00.083619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad3e1f-a811-4423-9e66-7359351cbf12 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-22c96738-3397-4f98-a6c4-8f6b1038469e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:00.091403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:00.093356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:00.095379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:00.095379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:00.095379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:00.095379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:00.099966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 260/493] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:00.101080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c96738-3397-4f98-a6c4-8f6b1038469e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:00.121030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 234/494] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:00.123363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad3e1f-a811-4423-9e66-7359351cbf12 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:00.207383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:00.234681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c96738-3397-4f98-a6c4-8f6b1038469e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:00.275316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:00.293228 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad3e1f-a811-4423-9e66-7359351cbf12 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:00.299317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d8fa8-ab39-4e80-a656-7d4e0f56388b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:00.300064 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d8fa8-ab39-4e80-a656-7d4e0f56388b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.300388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d8fa8-ab39-4e80-a656-7d4e0f56388b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.300739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d8fa8-ab39-4e80-a656-7d4e0f56388b None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.301587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d8fa8-ab39-4e80-a656-7d4e0f56388b None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:00.301955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d8fa8-ab39-4e80-a656-7d4e0f56388b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:00.303506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d8fa8-ab39-4e80-a656-7d4e0f56388b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:00.312942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 261/495] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:00.314312 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b55c62-55f8-4abf-ac0c-4f3862fdd870 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.314604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b55c62-55f8-4abf-ac0c-4f3862fdd870 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.314890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b55c62-55f8-4abf-ac0c-4f3862fdd870 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.316237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 262/496] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:00.317039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b55c62-55f8-4abf-ac0c-4f3862fdd870 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:00.333828 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72625d7-1963-4665-bddb-4dae3e224195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:00.334491 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72625d7-1963-4665-bddb-4dae3e224195 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.334778 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72625d7-1963-4665-bddb-4dae3e224195 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.335059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72625d7-1963-4665-bddb-4dae3e224195 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.335791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72625d7-1963-4665-bddb-4dae3e224195 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:00.336103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72625d7-1963-4665-bddb-4dae3e224195 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:00.337527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72625d7-1963-4665-bddb-4dae3e224195 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:00.350762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 235/497] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:00.358400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1fa80-8755-4b16-9594-1767a4d02a07 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.358400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1fa80-8755-4b16-9594-1767a4d02a07 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.358400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1fa80-8755-4b16-9594-1767a4d02a07 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.358400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 236/498] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:00.358400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1fa80-8755-4b16-9594-1767a4d02a07 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:00.450079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b55c62-55f8-4abf-ac0c-4f3862fdd870 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:00.477519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1fa80-8755-4b16-9594-1767a4d02a07 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:00.490589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c1817e-bbcd-422b-b86d-209c874602ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:00.491849 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1817e-bbcd-422b-b86d-209c874602ca None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.492325 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1817e-bbcd-422b-b86d-209c874602ca None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.492816 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c1817e-bbcd-422b-b86d-209c874602ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.505826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1817e-bbcd-422b-b86d-209c874602ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:00.508305 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c1817e-bbcd-422b-b86d-209c874602ca None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:00.525540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68c80d-e40e-40e2-9cae-eb2eaf7edba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:00.526792 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68c80d-e40e-40e2-9cae-eb2eaf7edba3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.527268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68c80d-e40e-40e2-9cae-eb2eaf7edba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.527698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68c80d-e40e-40e2-9cae-eb2eaf7edba3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-80c1817e-bbcd-422b-b86d-209c874602ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:00.537478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:00.538589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:00.539501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:00.539501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:00.539501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:00.539501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:00.539501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:00.539501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:00.539501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:00.539501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:00.539501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:42:00.539501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:00.540051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 263/499] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 23 11:42:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:00.541430 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf069a-518f-436d-b0d8-c03a5e3b3bcc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.541787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf069a-518f-436d-b0d8-c03a5e3b3bcc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.542108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf069a-518f-436d-b0d8-c03a5e3b3bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.551753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68c80d-e40e-40e2-9cae-eb2eaf7edba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:00.553279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68c80d-e40e-40e2-9cae-eb2eaf7edba3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:00.560620 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-afaf069a-518f-436d-b0d8-c03a5e3b3bcc None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:00.562421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-afaf069a-518f-436d-b0d8-c03a5e3b3bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:00.564824 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-7f68c80d-e40e-40e2-9cae-eb2eaf7edba3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:00.574386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:00.575351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:00.576216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:00.576216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:00.576216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:00.576216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:00.576216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 237/500] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:00.576486 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68c80d-e40e-40e2-9cae-eb2eaf7edba3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:00.580077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:00.651628 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 264/501] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:00.653974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb48155-e622-488e-92ec-164b7c4caee1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.654468 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb48155-e622-488e-92ec-164b7c4caee1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.654959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb48155-e622-488e-92ec-164b7c4caee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.681898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bcb48155-e622-488e-92ec-164b7c4caee1 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:00.683214 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bcb48155-e622-488e-92ec-164b7c4caee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:00.716774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68c80d-e40e-40e2-9cae-eb2eaf7edba3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:00.753880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ad7f4-3f94-4f2a-8b07-d9df4b05dee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:00.754607 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ad7f4-3f94-4f2a-8b07-d9df4b05dee8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.754891 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ad7f4-3f94-4f2a-8b07-d9df4b05dee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.755180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ad7f4-3f94-4f2a-8b07-d9df4b05dee8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.763345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 265/502] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:00.764048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ad7f4-3f94-4f2a-8b07-d9df4b05dee8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:00.764259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb48155-e622-488e-92ec-164b7c4caee1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:00.766236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ad7f4-3f94-4f2a-8b07-d9df4b05dee8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-5d4ad7f4-3f94-4f2a-8b07-d9df4b05dee8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:00.784895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:00.786515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:00.788217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:00.788217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:00.788217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:00.788217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:00.788217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:00.788217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:00.788217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:00.788217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:00.788217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:42:00.788217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:00.788217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 238/503] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:00.788217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ad7f4-3f94-4f2a-8b07-d9df4b05dee8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:00.862267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:00.864880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:00.911079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb48155-e622-488e-92ec-164b7c4caee1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:00.911843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ad7f4-3f94-4f2a-8b07-d9df4b05dee8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:00.954832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41531c5-7f98-42a4-b5b8-d2288138ab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:00.955517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41531c5-7f98-42a4-b5b8-d2288138ab91 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.955679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863eb51-d1f0-4029-ae0f-b899a62548af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:00.955812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41531c5-7f98-42a4-b5b8-d2288138ab91 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.956085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41531c5-7f98-42a4-b5b8-d2288138ab91 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.956813 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863eb51-d1f0-4029-ae0f-b899a62548af None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:00.956941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41531c5-7f98-42a4-b5b8-d2288138ab91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:00.957179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41531c5-7f98-42a4-b5b8-d2288138ab91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:00.957316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863eb51-d1f0-4029-ae0f-b899a62548af None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:00.957644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863eb51-d1f0-4029-ae0f-b899a62548af None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:00.958596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41531c5-7f98-42a4-b5b8-d2288138ab91 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:00.958929 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863eb51-d1f0-4029-ae0f-b899a62548af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:00.959293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863eb51-d1f0-4029-ae0f-b899a62548af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:00.960921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863eb51-d1f0-4029-ae0f-b899a62548af None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:00.972067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 239/504] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:00.973768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41531c5-7f98-42a4-b5b8-d2288138ab91 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:01.005777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 266/505] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 23 11:42:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:01.006584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863eb51-d1f0-4029-ae0f-b899a62548af None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:01.109145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41531c5-7f98-42a4-b5b8-d2288138ab91 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:01.143240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:01.145513 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863eb51-d1f0-4029-ae0f-b899a62548af None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:01.159255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59790936-7b8a-49d7-bf83-d837f0cb00d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:01.160063 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59790936-7b8a-49d7-bf83-d837f0cb00d4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:01.160346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59790936-7b8a-49d7-bf83-d837f0cb00d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:01.160625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59790936-7b8a-49d7-bf83-d837f0cb00d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:01.173757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59790936-7b8a-49d7-bf83-d837f0cb00d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-59790936-7b8a-49d7-bf83-d837f0cb00d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:01.181851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:01.183721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 240/506] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:01.185788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59790936-7b8a-49d7-bf83-d837f0cb00d4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:01.190967 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:01.202869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e943be7-02a7-4022-85a6-863b228657cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:01.203943 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e943be7-02a7-4022-85a6-863b228657cf None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:01.204407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e943be7-02a7-4022-85a6-863b228657cf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:01.204878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e943be7-02a7-4022-85a6-863b228657cf None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:01.206356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e943be7-02a7-4022-85a6-863b228657cf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:01.206883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e943be7-02a7-4022-85a6-863b228657cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:01.209343 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e943be7-02a7-4022-85a6-863b228657cf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:01.232968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 267/507] 10.222.0.67 () {56 vars in 1137 bytes} [Thu Jun 23 11:42:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:01.233974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e943be7-02a7-4022-85a6-863b228657cf None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:01.306791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59790936-7b8a-49d7-bf83-d837f0cb00d4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:01.343666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9ed3e-4232-433b-8417-361072c03a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:01.344619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9ed3e-4232-433b-8417-361072c03a43 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:01.344999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9ed3e-4232-433b-8417-361072c03a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:01.345288 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9ed3e-4232-433b-8417-361072c03a43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:01.348143 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e943be7-02a7-4022-85a6-863b228657cf None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:01.356082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9ed3e-4232-433b-8417-361072c03a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:01.357586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9ed3e-4232-433b-8417-361072c03a43 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-eed9ed3e-4232-433b-8417-361072c03a43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:01.378080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:01.379538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:01.380610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:01.380610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:01.380610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:01.380610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:01.384155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 241/508] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:01.385062 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9ed3e-4232-433b-8417-361072c03a43 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:01.395668 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af32a2f-8372-45de-8f50-b5ea748797f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:01.396464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af32a2f-8372-45de-8f50-b5ea748797f6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:01.396784 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af32a2f-8372-45de-8f50-b5ea748797f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:01.397071 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af32a2f-8372-45de-8f50-b5ea748797f6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:01.404517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af32a2f-8372-45de-8f50-b5ea748797f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:01.406040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af32a2f-8372-45de-8f50-b5ea748797f6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:01.420140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-1af32a2f-8372-45de-8f50-b5ea748797f6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:01.421178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:01.422134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:01.423045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:01.423045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:01.423045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:01.423045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:01.423045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:01.423045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:01.423045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:01.423045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:01.423045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 11:42:01.423045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:01.423045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 268/509] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:01.423690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af32a2f-8372-45de-8f50-b5ea748797f6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:01.471158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:01.496586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9ed3e-4232-433b-8417-361072c03a43 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:01.529801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af32a2f-8372-45de-8f50-b5ea748797f6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:01.533049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d73b3-6c1d-41da-bc82-607209d61f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:01.533710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d73b3-6c1d-41da-bc82-607209d61f97 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:01.534203 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d73b3-6c1d-41da-bc82-607209d61f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:01.534355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d73b3-6c1d-41da-bc82-607209d61f97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:01.568583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12a71d-b3db-4ec8-beb3-9d766d7f1d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:01.569536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12a71d-b3db-4ec8-beb3-9d766d7f1d88 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:01.569536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12a71d-b3db-4ec8-beb3-9d766d7f1d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:01.569536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12a71d-b3db-4ec8-beb3-9d766d7f1d88 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:01.570854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12a71d-b3db-4ec8-beb3-9d766d7f1d88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:01.571168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12a71d-b3db-4ec8-beb3-9d766d7f1d88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:01.572598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12a71d-b3db-4ec8-beb3-9d766d7f1d88 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:01.583948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 269/510] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:01.584870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12a71d-b3db-4ec8-beb3-9d766d7f1d88 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:01.627991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d73b3-6c1d-41da-bc82-607209d61f97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:01.629575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d73b3-6c1d-41da-bc82-607209d61f97 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:01.670799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:01.720046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12a71d-b3db-4ec8-beb3-9d766d7f1d88 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:01.729642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:01.757804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 242/511] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:42:01.758670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d73b3-6c1d-41da-bc82-607209d61f97 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:01.766974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5616e-5e77-4b39-ab95-73a9fdfc231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:01.768274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5616e-5e77-4b39-ab95-73a9fdfc231e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:01.768598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5616e-5e77-4b39-ab95-73a9fdfc231e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:01.768983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5616e-5e77-4b39-ab95-73a9fdfc231e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:01.850072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5616e-5e77-4b39-ab95-73a9fdfc231e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:01.851486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5616e-5e77-4b39-ab95-73a9fdfc231e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:01.887933 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d73b3-6c1d-41da-bc82-607209d61f97 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:01.932538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0454fa-7303-4062-bf56-c237a209dbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:01.933379 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0454fa-7303-4062-bf56-c237a209dbd6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:01.933808 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0454fa-7303-4062-bf56-c237a209dbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:01.934149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0454fa-7303-4062-bf56-c237a209dbd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:01.934990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0454fa-7303-4062-bf56-c237a209dbd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:01.935370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0454fa-7303-4062-bf56-c237a209dbd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:01.937647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0454fa-7303-4062-bf56-c237a209dbd6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:01.939975 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:01.952740 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 243/512] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:01.954145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0454fa-7303-4062-bf56-c237a209dbd6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:01.966578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 270/513] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:42:01.967465 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5616e-5e77-4b39-ab95-73a9fdfc231e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:01.978198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:02.067827 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0454fa-7303-4062-bf56-c237a209dbd6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:02.081749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5616e-5e77-4b39-ab95-73a9fdfc231e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:02.106303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c410bd-7606-4fee-8809-8fb1d3f3f105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:02.106858 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c410bd-7606-4fee-8809-8fb1d3f3f105 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:02.107139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c410bd-7606-4fee-8809-8fb1d3f3f105 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:02.107416 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c410bd-7606-4fee-8809-8fb1d3f3f105 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:02.108875 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c410bd-7606-4fee-8809-8fb1d3f3f105 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:02.109226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c410bd-7606-4fee-8809-8fb1d3f3f105 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:02.110649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c410bd-7606-4fee-8809-8fb1d3f3f105 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:02.125623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3353b58-8c9e-4b6a-a562-61c1204a4894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:02.126421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3353b58-8c9e-4b6a-a562-61c1204a4894 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:02.126705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3353b58-8c9e-4b6a-a562-61c1204a4894 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:02.126982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3353b58-8c9e-4b6a-a562-61c1204a4894 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:02.145629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3353b58-8c9e-4b6a-a562-61c1204a4894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:02.148485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3353b58-8c9e-4b6a-a562-61c1204a4894 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:02.160571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 244/514] 10.222.0.67 () {56 vars in 1137 bytes} [Thu Jun 23 11:42:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:02.162234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c410bd-7606-4fee-8809-8fb1d3f3f105 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-f3353b58-8c9e-4b6a-a562-61c1204a4894 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:02.180277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:02.181519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:02.182381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:02.182381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:02.182381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:02.182381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:02.182381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 271/515] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:02.182381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3353b58-8c9e-4b6a-a562-61c1204a4894 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:02.234143 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:02.250632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:02.290349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c410bd-7606-4fee-8809-8fb1d3f3f105 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:02.295376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3353b58-8c9e-4b6a-a562-61c1204a4894 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:02.325614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b368ac1-4cab-4865-8e8a-8a041b4cdc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:02.326404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b368ac1-4cab-4865-8e8a-8a041b4cdc54 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:02.326690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b368ac1-4cab-4865-8e8a-8a041b4cdc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:02.327002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b368ac1-4cab-4865-8e8a-8a041b4cdc54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:02.330729 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11000470-0395-466c-82d5-efbf6431206e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:02.331405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11000470-0395-466c-82d5-efbf6431206e None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:02.331695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11000470-0395-466c-82d5-efbf6431206e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:02.331977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11000470-0395-466c-82d5-efbf6431206e None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:02.332716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11000470-0395-466c-82d5-efbf6431206e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:02.334655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11000470-0395-466c-82d5-efbf6431206e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:02.337557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b368ac1-4cab-4865-8e8a-8a041b4cdc54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:02.339032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b368ac1-4cab-4865-8e8a-8a041b4cdc54 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:02.356399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 272/516] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:42:02 2022] POST /identity/v3/users => generated 247 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:02.357201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11000470-0395-466c-82d5-efbf6431206e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-2b368ac1-4cab-4865-8e8a-8a041b4cdc54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:02.360129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:02.361081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:02.361960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:02.361960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:02.361960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:02.361960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:02.361960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 245/517] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:02.361960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b368ac1-4cab-4865-8e8a-8a041b4cdc54 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:02.501790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11000470-0395-466c-82d5-efbf6431206e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:02.506271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b368ac1-4cab-4865-8e8a-8a041b4cdc54 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:02.508935 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:02.521845 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:02.551127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68e6f0-ae61-429a-a344-251a3b84835b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:02.552183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68e6f0-ae61-429a-a344-251a3b84835b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:02.552504 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68e6f0-ae61-429a-a344-251a3b84835b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:02.552761 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68e6f0-ae61-429a-a344-251a3b84835b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:02.555565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f94c0e-0171-46b4-870e-92a603a6f607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:02.556318 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f94c0e-0171-46b4-870e-92a603a6f607 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:02.556751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f94c0e-0171-46b4-870e-92a603a6f607 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:02.557056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f94c0e-0171-46b4-870e-92a603a6f607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:02.566789 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68e6f0-ae61-429a-a344-251a3b84835b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:02.568349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68e6f0-ae61-429a-a344-251a3b84835b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-fa68e6f0-ae61-429a-a344-251a3b84835b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:02.587524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:02.588506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:02.589572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:02.589572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:02.589572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:02.589572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:02.590194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 273/518] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:02.590931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68e6f0-ae61-429a-a344-251a3b84835b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:02.663450 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f94c0e-0171-46b4-870e-92a603a6f607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:02.665478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f94c0e-0171-46b4-870e-92a603a6f607 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:02.715072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68e6f0-ae61-429a-a344-251a3b84835b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:02.755324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841b4a5-7d56-40b4-997d-6207983cbb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:02.756000 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841b4a5-7d56-40b4-997d-6207983cbb7d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:02.756285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841b4a5-7d56-40b4-997d-6207983cbb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:02.756563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841b4a5-7d56-40b4-997d-6207983cbb7d None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:02.757336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841b4a5-7d56-40b4-997d-6207983cbb7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:02.757647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841b4a5-7d56-40b4-997d-6207983cbb7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:02.759076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6841b4a5-7d56-40b4-997d-6207983cbb7d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:02.773623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 274/519] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:02.775149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841b4a5-7d56-40b4-997d-6207983cbb7d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:02.788924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 246/520] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:42:02.790340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f94c0e-0171-46b4-870e-92a603a6f607 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:02.795012 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:02.817586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:02.895781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841b4a5-7d56-40b4-997d-6207983cbb7d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:02.901384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f94c0e-0171-46b4-870e-92a603a6f607 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:02.935805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad5fa8-d297-4b4c-a5f3-766632587efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:02.936399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad5fa8-d297-4b4c-a5f3-766632587efc None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:02.936707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad5fa8-d297-4b4c-a5f3-766632587efc None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:02.936992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad5fa8-d297-4b4c-a5f3-766632587efc None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:02.937722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad5fa8-d297-4b4c-a5f3-766632587efc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:02.938041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad5fa8-d297-4b4c-a5f3-766632587efc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:02.939419 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad5fa8-d297-4b4c-a5f3-766632587efc None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:02.940465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f39d93-da8d-4b47-9857-dd0de8e60a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:02.941273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f39d93-da8d-4b47-9857-dd0de8e60a7a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:02.941562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f39d93-da8d-4b47-9857-dd0de8e60a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:02.941949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f39d93-da8d-4b47-9857-dd0de8e60a7a None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:02.942663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f39d93-da8d-4b47-9857-dd0de8e60a7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:02.943157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f39d93-da8d-4b47-9857-dd0de8e60a7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:02.944528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f39d93-da8d-4b47-9857-dd0de8e60a7a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:02.951528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 275/521] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:02.951528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad5fa8-d297-4b4c-a5f3-766632587efc None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:02.964821 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 247/522] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:02.964821 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f39d93-da8d-4b47-9857-dd0de8e60a7a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:03.026029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:03.050479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:03.065708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad5fa8-d297-4b4c-a5f3-766632587efc None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:03.070502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f39d93-da8d-4b47-9857-dd0de8e60a7a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:03.101313 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bddc86-d853-4091-b293-facb831e2a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:03.102109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bddc86-d853-4091-b293-facb831e2a68 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.102394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bddc86-d853-4091-b293-facb831e2a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.102667 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bddc86-d853-4091-b293-facb831e2a68 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.105829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6f73d-1a9e-4474-9f29-2368e991b607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:03.106609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6f73d-1a9e-4474-9f29-2368e991b607 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.106892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6f73d-1a9e-4474-9f29-2368e991b607 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.107169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6f73d-1a9e-4474-9f29-2368e991b607 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.110904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bddc86-d853-4091-b293-facb831e2a68 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:03.112350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bddc86-d853-4091-b293-facb831e2a68 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:03.118943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6f73d-1a9e-4474-9f29-2368e991b607 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:03.120387 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6f73d-1a9e-4474-9f29-2368e991b607 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-63bddc86-d853-4091-b293-facb831e2a68 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:03.128097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:03.129029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:03.129918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:03.129918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:03.129918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:03.129918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:03.129918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:03.129918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:03.129918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:03.129918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:03.129918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 11:42:03.129918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:03.129918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 276/523] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:03.130617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bddc86-d853-4091-b293-facb831e2a68 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-80f6f73d-1a9e-4474-9f29-2368e991b607 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:03.145272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:03.146189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:03.147032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:03.147032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:03.147032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:03.147032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:03.152221 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 248/524] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:03.152221 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6f73d-1a9e-4474-9f29-2368e991b607 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:03.236624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bddc86-d853-4091-b293-facb831e2a68 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:03.258665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6f73d-1a9e-4474-9f29-2368e991b607 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:03.275174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:03.276113 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db8ada-bc0e-4c94-a641-228242c014bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:03.277053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db8ada-bc0e-4c94-a641-228242c014bf None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.277384 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db8ada-bc0e-4c94-a641-228242c014bf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.277990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db8ada-bc0e-4c94-a641-228242c014bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.286079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db8ada-bc0e-4c94-a641-228242c014bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:03.287554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db8ada-bc0e-4c94-a641-228242c014bf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:03.296870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9287e32-45da-4cd3-b10c-4696c60a100c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:03.297915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9287e32-45da-4cd3-b10c-4696c60a100c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.298372 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9287e32-45da-4cd3-b10c-4696c60a100c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.298928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9287e32-45da-4cd3-b10c-4696c60a100c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-76db8ada-bc0e-4c94-a641-228242c014bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:03.303110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:03.304040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:03.305163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:03.305163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:03.305163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:03.305163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:03.305163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:03.305163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:03.305163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:03.305163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:03.307681 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9287e32-45da-4cd3-b10c-4696c60a100c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:03.309077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 277/525] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:03.309363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9287e32-45da-4cd3-b10c-4696c60a100c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:03.310144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db8ada-bc0e-4c94-a641-228242c014bf None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:03.320691 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b9287e32-45da-4cd3-b10c-4696c60a100c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:03.329455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:03.330349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:03.331284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:03.331284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:03.331284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:03.331284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:03.331284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:03.331284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:03.331284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:03.331284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:03.331284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 249/526] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:03.331753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9287e32-45da-4cd3-b10c-4696c60a100c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:03.434281 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db8ada-bc0e-4c94-a641-228242c014bf None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:03.455200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9287e32-45da-4cd3-b10c-4696c60a100c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:03.474079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80527e24-0582-41a6-8648-5d654aeb38d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:03.474920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80527e24-0582-41a6-8648-5d654aeb38d6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.475349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80527e24-0582-41a6-8648-5d654aeb38d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.475489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80527e24-0582-41a6-8648-5d654aeb38d6 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.476371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80527e24-0582-41a6-8648-5d654aeb38d6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:03.476764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80527e24-0582-41a6-8648-5d654aeb38d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:03.478076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80527e24-0582-41a6-8648-5d654aeb38d6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:03.492367 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2ceba-6dc4-4ba5-aaab-00030d5079dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:03.493403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2ceba-6dc4-4ba5-aaab-00030d5079dc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.493403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2ceba-6dc4-4ba5-aaab-00030d5079dc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.493591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2ceba-6dc4-4ba5-aaab-00030d5079dc None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.494234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2ceba-6dc4-4ba5-aaab-00030d5079dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:03.494540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2ceba-6dc4-4ba5-aaab-00030d5079dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:03.495957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b2ceba-6dc4-4ba5-aaab-00030d5079dc None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:03.501764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 278/527] 10.222.0.67 () {56 vars in 1139 bytes} [Thu Jun 23 11:42:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:03.501764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80527e24-0582-41a6-8648-5d654aeb38d6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:03.512583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 250/528] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:03.512583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2ceba-6dc4-4ba5-aaab-00030d5079dc None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:03.549269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:03.572568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:03.615352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80527e24-0582-41a6-8648-5d654aeb38d6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:03.620534 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b2ceba-6dc4-4ba5-aaab-00030d5079dc None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:03.652713 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696dd123-9503-4c6f-9d89-8ccd090b452e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:03.653474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696dd123-9503-4c6f-9d89-8ccd090b452e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.653833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696dd123-9503-4c6f-9d89-8ccd090b452e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.654181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696dd123-9503-4c6f-9d89-8ccd090b452e None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.655082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696dd123-9503-4c6f-9d89-8ccd090b452e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:03.655294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84199ef-c8e5-4766-a826-7f980459c813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:03.655536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696dd123-9503-4c6f-9d89-8ccd090b452e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:03.658873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84199ef-c8e5-4766-a826-7f980459c813 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.658873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84199ef-c8e5-4766-a826-7f980459c813 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.658873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84199ef-c8e5-4766-a826-7f980459c813 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.659121 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696dd123-9503-4c6f-9d89-8ccd090b452e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:03.659220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84199ef-c8e5-4766-a826-7f980459c813 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:03.659220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84199ef-c8e5-4766-a826-7f980459c813 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:03.659728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84199ef-c8e5-4766-a826-7f980459c813 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:03.679271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 251/529] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:03.680116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84199ef-c8e5-4766-a826-7f980459c813 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:03.685971 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 279/530] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:03.687516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696dd123-9503-4c6f-9d89-8ccd090b452e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:03.784447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:03.791530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84199ef-c8e5-4766-a826-7f980459c813 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:03.803678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696dd123-9503-4c6f-9d89-8ccd090b452e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:03.829523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:03.830492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5108f8-2d63-46ef-be95-eb634d4a4258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:03.831280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5108f8-2d63-46ef-be95-eb634d4a4258 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.831615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5108f8-2d63-46ef-be95-eb634d4a4258 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.831965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5108f8-2d63-46ef-be95-eb634d4a4258 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.841525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fa797-8156-4f69-8a5b-e948a81facc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:03.842315 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fa797-8156-4f69-8a5b-e948a81facc1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:03.842599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fa797-8156-4f69-8a5b-e948a81facc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:03.842874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7fa797-8156-4f69-8a5b-e948a81facc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:03.850640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fa797-8156-4f69-8a5b-e948a81facc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:03.852215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7fa797-8156-4f69-8a5b-e948a81facc1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-cd7fa797-8156-4f69-8a5b-e948a81facc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:03.869459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:03.870388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:03.871332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:03.871332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:03.871332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:03.871332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:03.871332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:03.871332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:03.871332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:03.871332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:03.871332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 280/531] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:03.871819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fa797-8156-4f69-8a5b-e948a81facc1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:03.909199 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5108f8-2d63-46ef-be95-eb634d4a4258 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:03.910665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5108f8-2d63-46ef-be95-eb634d4a4258 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:03.984973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7fa797-8156-4f69-8a5b-e948a81facc1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:04.010190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:04.018456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 252/532] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:42:04.019326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5108f8-2d63-46ef-be95-eb634d4a4258 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:04.027333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e8ffb-25e2-404b-b2b9-aed37c228efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:04.028539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e8ffb-25e2-404b-b2b9-aed37c228efb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:04.028857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e8ffb-25e2-404b-b2b9-aed37c228efb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:04.029204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e8ffb-25e2-404b-b2b9-aed37c228efb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:04.030384 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e8ffb-25e2-404b-b2b9-aed37c228efb None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=abcbb404d9a148568a82697abd51ec30) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:04.030707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e8ffb-25e2-404b-b2b9-aed37c228efb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:04.032794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e8ffb-25e2-404b-b2b9-aed37c228efb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:04.039382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 281/533] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:03 2022] GET /identity/v3/role_assignments?user.id=abcbb404d9a148568a82697abd51ec30&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:04.051027 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e8ffb-25e2-404b-b2b9-aed37c228efb None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:04.133665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:04.147134 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5108f8-2d63-46ef-be95-eb634d4a4258 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:04.196479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f711686-df8c-41ea-a818-1da097f3128f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:04.197727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f711686-df8c-41ea-a818-1da097f3128f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:04.198419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f711686-df8c-41ea-a818-1da097f3128f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:04.198709 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f711686-df8c-41ea-a818-1da097f3128f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:04.199832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f711686-df8c-41ea-a818-1da097f3128f None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=36c18bbd22f74efbae7aab1cdd3f6403) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:04.200151 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f711686-df8c-41ea-a818-1da097f3128f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:04.201126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e8ffb-25e2-404b-b2b9-aed37c228efb None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:04.201636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f711686-df8c-41ea-a818-1da097f3128f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:04.209172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 253/534] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:04 2022] GET /identity/v3/role_assignments?user.id=36c18bbd22f74efbae7aab1cdd3f6403&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:04.210337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f711686-df8c-41ea-a818-1da097f3128f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:04.242431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff00f7-8d18-43d3-87e0-e515dc3555b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:04.243284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff00f7-8d18-43d3-87e0-e515dc3555b2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:04.243571 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff00f7-8d18-43d3-87e0-e515dc3555b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:04.243844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ff00f7-8d18-43d3-87e0-e515dc3555b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:04.244640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff00f7-8d18-43d3-87e0-e515dc3555b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:04.245001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff00f7-8d18-43d3-87e0-e515dc3555b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:04.246468 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ff00f7-8d18-43d3-87e0-e515dc3555b2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:04.261298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 282/535] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:04.267632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff00f7-8d18-43d3-87e0-e515dc3555b2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:04.297981 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:04.353370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f711686-df8c-41ea-a818-1da097f3128f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:04.428797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcba91-e5b5-466e-85e9-850f31f6f7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:04.429614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcba91-e5b5-466e-85e9-850f31f6f7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:04.429899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcba91-e5b5-466e-85e9-850f31f6f7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:04.430179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcba91-e5b5-466e-85e9-850f31f6f7e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:04.441571 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ff00f7-8d18-43d3-87e0-e515dc3555b2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:04.443694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcba91-e5b5-466e-85e9-850f31f6f7e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:04.445358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcba91-e5b5-466e-85e9-850f31f6f7e3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:04.452850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-a7dcba91-e5b5-466e-85e9-850f31f6f7e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:04.467380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:04.468878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:04.469711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:04.469711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:04.469711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:04.469711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:04.469711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 254/536] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:04.475051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcba91-e5b5-466e-85e9-850f31f6f7e3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:04.485161 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728abf7c-d3ed-406d-8f30-2485676a33e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:04.485831 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728abf7c-d3ed-406d-8f30-2485676a33e3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:04.486125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728abf7c-d3ed-406d-8f30-2485676a33e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:04.486404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728abf7c-d3ed-406d-8f30-2485676a33e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:04.487250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728abf7c-d3ed-406d-8f30-2485676a33e3 None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=74fdc12d728d45018ea668a4073da3b8) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:04.487569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728abf7c-d3ed-406d-8f30-2485676a33e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:04.489340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728abf7c-d3ed-406d-8f30-2485676a33e3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:04.495839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 283/537] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:04 2022] GET /identity/v3/role_assignments?user.id=74fdc12d728d45018ea668a4073da3b8&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:04.501439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd2117-bb09-44d0-8cea-daa467af3217 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:04.501745 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd2117-bb09-44d0-8cea-daa467af3217 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:04.502015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dd2117-bb09-44d0-8cea-daa467af3217 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:04.503791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 284/538] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:04.510464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cbc39-2f0f-446a-a6f6-f84686078d71 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:04.510825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cbc39-2f0f-446a-a6f6-f84686078d71 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:04.511116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249cbc39-2f0f-446a-a6f6-f84686078d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:04.558035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-249cbc39-2f0f-446a-a6f6-f84686078d71 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:04.559240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-249cbc39-2f0f-446a-a6f6-f84686078d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:04.594817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcba91-e5b5-466e-85e9-850f31f6f7e3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:04.612165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:04.638772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15b4e9-8e51-4d4d-9e9c-0c4bd3d66b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:04.639502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15b4e9-8e51-4d4d-9e9c-0c4bd3d66b1d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:04.639793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15b4e9-8e51-4d4d-9e9c-0c4bd3d66b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:04.640076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15b4e9-8e51-4d4d-9e9c-0c4bd3d66b1d None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:04.640896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15b4e9-8e51-4d4d-9e9c-0c4bd3d66b1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:04.641220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15b4e9-8e51-4d4d-9e9c-0c4bd3d66b1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:04.642938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15b4e9-8e51-4d4d-9e9c-0c4bd3d66b1d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:04.657929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 255/539] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:04.664283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15b4e9-8e51-4d4d-9e9c-0c4bd3d66b1d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:04.674915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 285/540] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:04.709602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cbc39-2f0f-446a-a6f6-f84686078d71 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:04.797909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15b4e9-8e51-4d4d-9e9c-0c4bd3d66b1d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:04.854746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb0f0b-c326-4e10-8d06-b187c7a86786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:04.858138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb0f0b-c326-4e10-8d06-b187c7a86786 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:04.860303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb0f0b-c326-4e10-8d06-b187c7a86786 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:04.860892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb0f0b-c326-4e10-8d06-b187c7a86786 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:04.879223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb0f0b-c326-4e10-8d06-b187c7a86786 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:04.881223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb0f0b-c326-4e10-8d06-b187c7a86786 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-c5fb0f0b-c326-4e10-8d06-b187c7a86786 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:04.911934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:04.912894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:04.913782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:04.913782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:04.913782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:04.913782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:04.913782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:04.913782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:04.913782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:04.913782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:04.918196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 256/541] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:04.937776 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb0f0b-c326-4e10-8d06-b187c7a86786 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:04.946571 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249cbc39-2f0f-446a-a6f6-f84686078d71 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:04.949067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:05.021661 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18ee7e-28c7-47a7-85f4-8e00a864670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:05.022907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18ee7e-28c7-47a7-85f4-8e00a864670c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:05.024460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18ee7e-28c7-47a7-85f4-8e00a864670c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:05.025063 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd18ee7e-28c7-47a7-85f4-8e00a864670c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:05.033667 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:05.057648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd18ee7e-28c7-47a7-85f4-8e00a864670c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-dd18ee7e-28c7-47a7-85f4-8e00a864670c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:05.072183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:05.073197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:05.074013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:05.074013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:05.074013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:05.074013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:05.074013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:05.074013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:05.074013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:05.074013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:05.074013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:05.074013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:05.074013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:05.074013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:05.082314 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 286/542] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:05.086664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18ee7e-28c7-47a7-85f4-8e00a864670c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:05.234294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb0f0b-c326-4e10-8d06-b187c7a86786 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:05.236581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd18ee7e-28c7-47a7-85f4-8e00a864670c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:05.282349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2e8ce-7a1c-4108-b120-2f53fe0086f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:05.283383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2e8ce-7a1c-4108-b120-2f53fe0086f0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:05.285011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2e8ce-7a1c-4108-b120-2f53fe0086f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:05.288847 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a2e8ce-7a1c-4108-b120-2f53fe0086f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:05.289947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2e8ce-7a1c-4108-b120-2f53fe0086f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:05.290474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2e8ce-7a1c-4108-b120-2f53fe0086f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:05.292119 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a2e8ce-7a1c-4108-b120-2f53fe0086f0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:05.312817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d80302-e307-4d97-bd1f-62e3501718dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:05.314045 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d80302-e307-4d97-bd1f-62e3501718dc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:05.314556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d80302-e307-4d97-bd1f-62e3501718dc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:05.315038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d80302-e307-4d97-bd1f-62e3501718dc None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:05.316039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d80302-e307-4d97-bd1f-62e3501718dc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:05.316557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d80302-e307-4d97-bd1f-62e3501718dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:05.318402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d80302-e307-4d97-bd1f-62e3501718dc None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:05.327670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 287/543] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:05.337813 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2e8ce-7a1c-4108-b120-2f53fe0086f0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:05.339112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 257/544] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:05.349983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d80302-e307-4d97-bd1f-62e3501718dc None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:05.397072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:05.582941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a2e8ce-7a1c-4108-b120-2f53fe0086f0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:05.608704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:05.616596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d80302-e307-4d97-bd1f-62e3501718dc None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:05.650979 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd38d8a-fd1a-41fb-b202-4495b2e36ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:05.651790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd38d8a-fd1a-41fb-b202-4495b2e36ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:05.652080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd38d8a-fd1a-41fb-b202-4495b2e36ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:05.652361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd38d8a-fd1a-41fb-b202-4495b2e36ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:05.658269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c738e6-02fc-476c-a0d5-77270c429eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:05.658939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c738e6-02fc-476c-a0d5-77270c429eec None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:05.659229 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c738e6-02fc-476c-a0d5-77270c429eec None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:05.659512 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c738e6-02fc-476c-a0d5-77270c429eec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:05.660355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c738e6-02fc-476c-a0d5-77270c429eec None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=5f095d3370be457cb6687ac8f8c9f8ab) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:05.660701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c738e6-02fc-476c-a0d5-77270c429eec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:05.662149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c738e6-02fc-476c-a0d5-77270c429eec None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:05.677203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd38d8a-fd1a-41fb-b202-4495b2e36ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:05.678677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd38d8a-fd1a-41fb-b202-4495b2e36ce4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:05.680142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 258/545] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:05 2022] GET /identity/v3/role_assignments?user.id=5f095d3370be457cb6687ac8f8c9f8ab&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8bd38d8a-fd1a-41fb-b202-4495b2e36ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:05.699556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:05.701664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:05.703587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:05.703587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:05.703587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:05.703587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:05.703587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 288/546] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:05.713210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c738e6-02fc-476c-a0d5-77270c429eec None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:05.833258 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c738e6-02fc-476c-a0d5-77270c429eec None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:05.876843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02a437-9688-4641-ad26-2083ae69c59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:05.877525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02a437-9688-4641-ad26-2083ae69c59c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:05.877798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02a437-9688-4641-ad26-2083ae69c59c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:05.878075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02a437-9688-4641-ad26-2083ae69c59c None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:05.878811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02a437-9688-4641-ad26-2083ae69c59c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:05.879123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02a437-9688-4641-ad26-2083ae69c59c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:05.880503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc02a437-9688-4641-ad26-2083ae69c59c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:05.893825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 259/547] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:05.903283 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd38d8a-fd1a-41fb-b202-4495b2e36ce4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:06.018732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:06.048049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd38d8a-fd1a-41fb-b202-4495b2e36ce4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:06.087663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f99088-3021-4666-b009-ebbceafe3988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:06.088387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f99088-3021-4666-b009-ebbceafe3988 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:06.088686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f99088-3021-4666-b009-ebbceafe3988 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:06.088985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f99088-3021-4666-b009-ebbceafe3988 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:06.097958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f99088-3021-4666-b009-ebbceafe3988 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:06.099512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f99088-3021-4666-b009-ebbceafe3988 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e4f99088-3021-4666-b009-ebbceafe3988 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:06.115037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:06.115996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:06.117440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:06.117440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:06.117440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:06.117440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:06.117440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:06.117440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:06.117440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:06.117440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:06.117440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:42:06.117440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:06.120911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 289/548] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:42:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:06.122337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02a437-9688-4641-ad26-2083ae69c59c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:06.154172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:06.251222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02a437-9688-4641-ad26-2083ae69c59c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:06.301998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af55a9-d6d6-4402-90ee-b4e01d8843af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:06.302680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af55a9-d6d6-4402-90ee-b4e01d8843af None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:06.302961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af55a9-d6d6-4402-90ee-b4e01d8843af None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:06.303239 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af55a9-d6d6-4402-90ee-b4e01d8843af None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:06.304145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af55a9-d6d6-4402-90ee-b4e01d8843af None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:06.304459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af55a9-d6d6-4402-90ee-b4e01d8843af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:06.306013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af55a9-d6d6-4402-90ee-b4e01d8843af None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:06.333687 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 260/549] 10.222.0.67 () {56 vars in 1135 bytes} [Thu Jun 23 11:42:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:06.339435 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f99088-3021-4666-b009-ebbceafe3988 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:06.464658 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f99088-3021-4666-b009-ebbceafe3988 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:06.507694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cad531-b81a-4c0b-be83-b416b51dc696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:06.508396 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cad531-b81a-4c0b-be83-b416b51dc696 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:06.508697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cad531-b81a-4c0b-be83-b416b51dc696 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:06.508985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cad531-b81a-4c0b-be83-b416b51dc696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:06.517707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:06.589560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cad531-b81a-4c0b-be83-b416b51dc696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:06.590944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cad531-b81a-4c0b-be83-b416b51dc696 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:06.711941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 290/550] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:42:06.717123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af55a9-d6d6-4402-90ee-b4e01d8843af None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:06.765577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a2c8e-762d-4642-82a9-c31ae7e6f118 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:06.765910 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a2c8e-762d-4642-82a9-c31ae7e6f118 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:06.766137 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a2c8e-762d-4642-82a9-c31ae7e6f118 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:06.767837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 291/551] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:06.773069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb538dc-9421-4fa2-ae83-917dda7944f1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:06.773355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb538dc-9421-4fa2-ae83-917dda7944f1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:06.773629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb538dc-9421-4fa2-ae83-917dda7944f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:06.798996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1bb538dc-9421-4fa2-ae83-917dda7944f1 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:06.800189 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1bb538dc-9421-4fa2-ae83-917dda7944f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:06.857131 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:06.857300 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:06.902407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af55a9-d6d6-4402-90ee-b4e01d8843af None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:06.926074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 292/552] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:06.952955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d678c47-77e3-467d-a2aa-a5d17e67cb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:06.953867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d678c47-77e3-467d-a2aa-a5d17e67cb70 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:06.954151 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d678c47-77e3-467d-a2aa-a5d17e67cb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:06.954427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d678c47-77e3-467d-a2aa-a5d17e67cb70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:06.960465 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb538dc-9421-4fa2-ae83-917dda7944f1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:06.982462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d678c47-77e3-467d-a2aa-a5d17e67cb70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:06.983996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d678c47-77e3-467d-a2aa-a5d17e67cb70 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-8d678c47-77e3-467d-a2aa-a5d17e67cb70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:07.016934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:07.018006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:07.018879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:07.018879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:07.018879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:07.018879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:07.021244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 261/553] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:07.025973 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d678c47-77e3-467d-a2aa-a5d17e67cb70 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:07.133014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb538dc-9421-4fa2-ae83-917dda7944f1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:07.173837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad242f-9efa-4bce-9e50-479cae0b3590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:07.174787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad242f-9efa-4bce-9e50-479cae0b3590 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:07.175073 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad242f-9efa-4bce-9e50-479cae0b3590 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:07.175349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad242f-9efa-4bce-9e50-479cae0b3590 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:07.187191 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d678c47-77e3-467d-a2aa-a5d17e67cb70 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:07.187367 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad242f-9efa-4bce-9e50-479cae0b3590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2fad242f-9efa-4bce-9e50-479cae0b3590 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:07.195106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:07.196046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:07.196893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:07.196893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:07.196893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:07.196893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:07.196893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:07.196893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:07.196893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:07.196893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:07.196893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:07.196893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:07.196893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:07.196893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:07.200920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 293/554] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:07.202910 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2b3c0-ad05-41b7-a380-84305649d289 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:07.203200 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2b3c0-ad05-41b7-a380-84305649d289 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:07.203481 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2b3c0-ad05-41b7-a380-84305649d289 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:07.209017 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 294/555] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:07.209811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2b3c0-ad05-41b7-a380-84305649d289 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:07.222151 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:07.225719 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0c794-8d9e-4648-bec1-331f552cfca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:07.226412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0c794-8d9e-4648-bec1-331f552cfca7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:07.226746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0c794-8d9e-4648-bec1-331f552cfca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:07.227206 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0c794-8d9e-4648-bec1-331f552cfca7 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:07.227976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0c794-8d9e-4648-bec1-331f552cfca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:07.228359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0c794-8d9e-4648-bec1-331f552cfca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:07.230068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0c794-8d9e-4648-bec1-331f552cfca7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:07.241311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:07.244611 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 262/556] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:07.245986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984218f2-2636-4148-ad98-8a70cefea229 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:07.246282 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984218f2-2636-4148-ad98-8a70cefea229 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:07.246573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984218f2-2636-4148-ad98-8a70cefea229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:07.275600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-984218f2-2636-4148-ad98-8a70cefea229 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:07.277094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-984218f2-2636-4148-ad98-8a70cefea229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:07.353075 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2b3c0-ad05-41b7-a380-84305649d289 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:07.359477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 263/557] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:07.360629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984218f2-2636-4148-ad98-8a70cefea229 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:07.395640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398572b-355b-4c80-8f05-5ea31df1231e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:07.396231 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398572b-355b-4c80-8f05-5ea31df1231e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:07.396517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398572b-355b-4c80-8f05-5ea31df1231e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:07.396833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398572b-355b-4c80-8f05-5ea31df1231e None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:07.397585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398572b-355b-4c80-8f05-5ea31df1231e None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:07.397893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398572b-355b-4c80-8f05-5ea31df1231e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:07.399748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398572b-355b-4c80-8f05-5ea31df1231e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:07.409635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 295/558] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:07.411159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678444-788f-457a-8d4f-cb7f4807475c None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:07.411159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678444-788f-457a-8d4f-cb7f4807475c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:07.411709 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678444-788f-457a-8d4f-cb7f4807475c None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:07.418055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 296/559] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:07.418055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba678444-788f-457a-8d4f-cb7f4807475c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:07.508574 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984218f2-2636-4148-ad98-8a70cefea229 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:07.521597 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:07.532857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:07.535585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba678444-788f-457a-8d4f-cb7f4807475c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:07.553742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a33c66-2b31-48da-aebf-ef4085c223d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:07.554551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a33c66-2b31-48da-aebf-ef4085c223d1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:07.554845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a33c66-2b31-48da-aebf-ef4085c223d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:07.555223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a33c66-2b31-48da-aebf-ef4085c223d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:07.563589 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a33c66-2b31-48da-aebf-ef4085c223d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:07.565262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a33c66-2b31-48da-aebf-ef4085c223d1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:07.577791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca251ff-ca55-49dd-8498-3f5d015a6403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:07.578716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca251ff-ca55-49dd-8498-3f5d015a6403 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:07.579070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca251ff-ca55-49dd-8498-3f5d015a6403 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:07.579345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca251ff-ca55-49dd-8498-3f5d015a6403 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-62a33c66-2b31-48da-aebf-ef4085c223d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:07.581562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:07.582494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:07.588234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:07.588234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:07.588234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:07.588234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:07.588234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:07.588234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:07.588234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:07.588234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:07.588234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 264/560] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:07.588234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a33c66-2b31-48da-aebf-ef4085c223d1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:07.604972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca251ff-ca55-49dd-8498-3f5d015a6403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2ca251ff-ca55-49dd-8498-3f5d015a6403 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:07.613006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:07.614100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:07.614933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 297/561] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:07.615663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be473b2-b575-46a7-a416-bb354c5d2627 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:07.616115 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be473b2-b575-46a7-a416-bb354c5d2627 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:07.616115 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be473b2-b575-46a7-a416-bb354c5d2627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:07.634558 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0be473b2-b575-46a7-a416-bb354c5d2627 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:07.635786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0be473b2-b575-46a7-a416-bb354c5d2627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:07.728855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 298/562] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:07.730120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be473b2-b575-46a7-a416-bb354c5d2627 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:07.730232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a33c66-2b31-48da-aebf-ef4085c223d1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:07.773845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61094a0-3277-4236-ae10-df4f5cedb2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:07.775056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61094a0-3277-4236-ae10-df4f5cedb2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:07.775056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61094a0-3277-4236-ae10-df4f5cedb2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:07.775056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61094a0-3277-4236-ae10-df4f5cedb2c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:07.796629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61094a0-3277-4236-ae10-df4f5cedb2c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:07.798118 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61094a0-3277-4236-ae10-df4f5cedb2c2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-e61094a0-3277-4236-ae10-df4f5cedb2c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:07.824238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:07.825279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:07.826332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:07.826332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:07.826332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:07.826332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:07.830117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 265/563] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:07.831023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61094a0-3277-4236-ae10-df4f5cedb2c2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:07.845683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:07.862514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:07.885325 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be473b2-b575-46a7-a416-bb354c5d2627 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:07.923563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de818c-812f-4d97-a03c-8aa671ad63cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:07.924144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de818c-812f-4d97-a03c-8aa671ad63cf None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:07.924426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de818c-812f-4d97-a03c-8aa671ad63cf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:07.924725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de818c-812f-4d97-a03c-8aa671ad63cf None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:07.925524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de818c-812f-4d97-a03c-8aa671ad63cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:07.925894 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de818c-812f-4d97-a03c-8aa671ad63cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:07.927383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de818c-812f-4d97-a03c-8aa671ad63cf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:07.945006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 299/564] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:07.945006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de818c-812f-4d97-a03c-8aa671ad63cf None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:07.952257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61094a0-3277-4236-ae10-df4f5cedb2c2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:07.991252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a31b5-6d98-4dfd-8951-f0315d0f1a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:07.991929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a31b5-6d98-4dfd-8951-f0315d0f1a2c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:07.992221 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a31b5-6d98-4dfd-8951-f0315d0f1a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:07.992503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a31b5-6d98-4dfd-8951-f0315d0f1a2c None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:07.993293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a31b5-6d98-4dfd-8951-f0315d0f1a2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:07.993606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a31b5-6d98-4dfd-8951-f0315d0f1a2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:07.994980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a31b5-6d98-4dfd-8951-f0315d0f1a2c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:08.008633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 266/565] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:08.008633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a31b5-6d98-4dfd-8951-f0315d0f1a2c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:08.068011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de818c-812f-4d97-a03c-8aa671ad63cf None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:08.096419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:08.103100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294b6d3-60e4-4623-baa4-5c0342be556e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:08.103895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294b6d3-60e4-4623-baa4-5c0342be556e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:08.104185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294b6d3-60e4-4623-baa4-5c0342be556e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:08.104461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294b6d3-60e4-4623-baa4-5c0342be556e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:08.106604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a31b5-6d98-4dfd-8951-f0315d0f1a2c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:08.113045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:08.117756 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294b6d3-60e4-4623-baa4-5c0342be556e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-d294b6d3-60e4-4623-baa4-5c0342be556e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:08.126842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:08.127884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:08.128778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 300/566] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:08.129468 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294b6d3-60e4-4623-baa4-5c0342be556e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:08.152525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4c79e-7b46-42cc-8ab2-14dab7b575aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:08.153608 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4c79e-7b46-42cc-8ab2-14dab7b575aa None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:08.154024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4c79e-7b46-42cc-8ab2-14dab7b575aa None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:08.154487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4c79e-7b46-42cc-8ab2-14dab7b575aa None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:08.155800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4c79e-7b46-42cc-8ab2-14dab7b575aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:08.156541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4c79e-7b46-42cc-8ab2-14dab7b575aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:08.159059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4c79e-7b46-42cc-8ab2-14dab7b575aa None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:08.182600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 267/567] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:08.183840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4c79e-7b46-42cc-8ab2-14dab7b575aa None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:08.259319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294b6d3-60e4-4623-baa4-5c0342be556e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:08.291336 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4c79e-7b46-42cc-8ab2-14dab7b575aa None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:08.303229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935da46a-7583-459a-bf24-4334027f954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:08.304097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935da46a-7583-459a-bf24-4334027f954b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:08.304387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935da46a-7583-459a-bf24-4334027f954b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:08.307258 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935da46a-7583-459a-bf24-4334027f954b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:08.307258 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935da46a-7583-459a-bf24-4334027f954b None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=eb9b9af5e0b44e34837e1cdb5ff6d33e) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:08.307258 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935da46a-7583-459a-bf24-4334027f954b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:08.307258 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935da46a-7583-459a-bf24-4334027f954b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:08.313516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 301/568] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:08 2022] GET /identity/v3/role_assignments?user.id=eb9b9af5e0b44e34837e1cdb5ff6d33e&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:08.314815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7fcfc-3c7f-47cf-9961-2d1d7ab62bff None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:08.315105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7fcfc-3c7f-47cf-9961-2d1d7ab62bff None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:08.315388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7fcfc-3c7f-47cf-9961-2d1d7ab62bff None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:08.316707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 302/569] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:08.317436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7fcfc-3c7f-47cf-9961-2d1d7ab62bff None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:08.332310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ea86f-8a66-4561-b249-776c115e57b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:08.333253 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ea86f-8a66-4561-b249-776c115e57b3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:08.333618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ea86f-8a66-4561-b249-776c115e57b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:08.333961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ea86f-8a66-4561-b249-776c115e57b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:08.345482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ea86f-8a66-4561-b249-776c115e57b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:08.347094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4ea86f-8a66-4561-b249-776c115e57b3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-ec4ea86f-8a66-4561-b249-776c115e57b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:08.366227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:08.367209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:08.368087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:08.368087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:08.368087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:08.368087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:08.368087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 268/570] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:08.368372 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ea86f-8a66-4561-b249-776c115e57b3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:08.375462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:08.406483 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:08.455459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7fcfc-3c7f-47cf-9961-2d1d7ab62bff None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:08.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ea86f-8a66-4561-b249-776c115e57b3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:08.497133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51bee92-d6de-45c4-b45f-0b5cefaf7625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:08.497808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bee92-d6de-45c4-b45f-0b5cefaf7625 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:08.498095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bee92-d6de-45c4-b45f-0b5cefaf7625 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:08.498375 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51bee92-d6de-45c4-b45f-0b5cefaf7625 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:08.499724 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bee92-d6de-45c4-b45f-0b5cefaf7625 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:08.500031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bee92-d6de-45c4-b45f-0b5cefaf7625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:08.501434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51bee92-d6de-45c4-b45f-0b5cefaf7625 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:08.511199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 303/571] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:08.517095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014aef5e-c63c-4c2f-a9a9-88572f92537e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:08.517397 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014aef5e-c63c-4c2f-a9a9-88572f92537e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:08.517684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014aef5e-c63c-4c2f-a9a9-88572f92537e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:08.532384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c71960-95e5-46ec-bb30-16eb0e18b10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:08.533268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c71960-95e5-46ec-bb30-16eb0e18b10f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:08.533606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c71960-95e5-46ec-bb30-16eb0e18b10f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:08.533881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c71960-95e5-46ec-bb30-16eb0e18b10f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:08.535481 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-014aef5e-c63c-4c2f-a9a9-88572f92537e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:08.536713 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-014aef5e-c63c-4c2f-a9a9-88572f92537e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:08.541409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c71960-95e5-46ec-bb30-16eb0e18b10f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:08.542850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c71960-95e5-46ec-bb30-16eb0e18b10f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b5c71960-95e5-46ec-bb30-16eb0e18b10f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:08.558395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:08.559360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:08.560351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:08.560351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:08.560351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:08.560351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:08.560351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:08.560351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:08.560351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:08.560351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:08.560351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:42:08.560351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:08.560351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 269/572] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:08.560351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c71960-95e5-46ec-bb30-16eb0e18b10f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:08.621481 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 304/573] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:08.622429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014aef5e-c63c-4c2f-a9a9-88572f92537e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:08.633090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:08.685081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:08.688315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c71960-95e5-46ec-bb30-16eb0e18b10f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:08.724792 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6e88-c853-4d46-9947-015740c9c125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:08.725461 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6e88-c853-4d46-9947-015740c9c125 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:08.725750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6e88-c853-4d46-9947-015740c9c125 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:08.726030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfd6e88-c853-4d46-9947-015740c9c125 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:08.726766 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd6e88-c853-4d46-9947-015740c9c125 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:08.727077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd6e88-c853-4d46-9947-015740c9c125 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:08.728448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfd6e88-c853-4d46-9947-015740c9c125 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:08.736018 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014aef5e-c63c-4c2f-a9a9-88572f92537e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:08.744880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 270/574] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:08.744880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6e88-c853-4d46-9947-015740c9c125 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:08.778135 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92e144-1e25-40c1-bda6-856926d35321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:08.778934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92e144-1e25-40c1-bda6-856926d35321 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:08.779220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92e144-1e25-40c1-bda6-856926d35321 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:08.779502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa92e144-1e25-40c1-bda6-856926d35321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:08.791471 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92e144-1e25-40c1-bda6-856926d35321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:08.792965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa92e144-1e25-40c1-bda6-856926d35321 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-fa92e144-1e25-40c1-bda6-856926d35321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:08.812052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:08.813207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:08.818791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:08.818791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:08.818791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:08.818791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:08.818791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 305/575] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:08.818791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92e144-1e25-40c1-bda6-856926d35321 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:08.858697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfd6e88-c853-4d46-9947-015740c9c125 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:08.882136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:08.896277 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950991bf-7962-49b0-8fed-33e2458daceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:08.896987 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950991bf-7962-49b0-8fed-33e2458daceb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:08.897276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950991bf-7962-49b0-8fed-33e2458daceb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:08.897557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950991bf-7962-49b0-8fed-33e2458daceb None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:08.898401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950991bf-7962-49b0-8fed-33e2458daceb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:08.898713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950991bf-7962-49b0-8fed-33e2458daceb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:08.900134 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950991bf-7962-49b0-8fed-33e2458daceb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:08.918173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 271/576] 10.222.0.67 () {56 vars in 1137 bytes} [Thu Jun 23 11:42:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:08.919004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950991bf-7962-49b0-8fed-33e2458daceb None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:08.935689 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa92e144-1e25-40c1-bda6-856926d35321 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:08.943609 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:08.976639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82be6b-edeb-4c0e-96f1-4e546c8a5f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:08.977353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82be6b-edeb-4c0e-96f1-4e546c8a5f73 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:08.977762 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82be6b-edeb-4c0e-96f1-4e546c8a5f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:08.978045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82be6b-edeb-4c0e-96f1-4e546c8a5f73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:08.990554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82be6b-edeb-4c0e-96f1-4e546c8a5f73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3c82be6b-edeb-4c0e-96f1-4e546c8a5f73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:08.998852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:08.999825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 306/577] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:09.001344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82be6b-edeb-4c0e-96f1-4e546c8a5f73 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:09.032419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950991bf-7962-49b0-8fed-33e2458daceb None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:09.071098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437fd46-cd3c-443d-a7d9-19702c4578d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:09.071893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437fd46-cd3c-443d-a7d9-19702c4578d0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:09.072181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437fd46-cd3c-443d-a7d9-19702c4578d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:09.072459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437fd46-cd3c-443d-a7d9-19702c4578d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:09.079814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437fd46-cd3c-443d-a7d9-19702c4578d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:09.081348 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437fd46-cd3c-443d-a7d9-19702c4578d0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-5437fd46-cd3c-443d-a7d9-19702c4578d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:09.096701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:09.097924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:09.098909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:09.098909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:09.098909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:09.098909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:09.098909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:09.098909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:09.098909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:09.098909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:09.098909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:42:09.098909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:09.098909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 272/578] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 23 11:42:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:09.099505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437fd46-cd3c-443d-a7d9-19702c4578d0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:09.116856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82be6b-edeb-4c0e-96f1-4e546c8a5f73 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:09.143432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:09.154459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ceb6e-7c2b-4137-8964-7a8764527060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:09.155861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ceb6e-7c2b-4137-8964-7a8764527060 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:09.156294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ceb6e-7c2b-4137-8964-7a8764527060 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:09.156756 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ceb6e-7c2b-4137-8964-7a8764527060 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:09.157737 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ceb6e-7c2b-4137-8964-7a8764527060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:09.158048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ceb6e-7c2b-4137-8964-7a8764527060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:09.159423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ceb6e-7c2b-4137-8964-7a8764527060 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:09.176817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 307/579] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:09.176817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ceb6e-7c2b-4137-8964-7a8764527060 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:09.204509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:09.212446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437fd46-cd3c-443d-a7d9-19702c4578d0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:09.258473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317bdd5-36c2-490e-be55-85b573f969f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:09.259710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317bdd5-36c2-490e-be55-85b573f969f8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:09.260164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317bdd5-36c2-490e-be55-85b573f969f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:09.260621 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317bdd5-36c2-490e-be55-85b573f969f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:09.276566 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317bdd5-36c2-490e-be55-85b573f969f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:09.278919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317bdd5-36c2-490e-be55-85b573f969f8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-8317bdd5-36c2-490e-be55-85b573f969f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:09.309631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:09.310609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:09.311555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:09.311555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:09.311555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:09.311555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:09.311785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ceb6e-7c2b-4137-8964-7a8764527060 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:09.317024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 273/580] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:09.318517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317bdd5-36c2-490e-be55-85b573f969f8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:09.352846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcab18-7128-4bca-b9c9-2199f412a61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:09.353514 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcab18-7128-4bca-b9c9-2199f412a61f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:09.353800 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcab18-7128-4bca-b9c9-2199f412a61f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:09.354079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcab18-7128-4bca-b9c9-2199f412a61f None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:09.354840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcab18-7128-4bca-b9c9-2199f412a61f None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:09.355155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcab18-7128-4bca-b9c9-2199f412a61f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:09.356541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcab18-7128-4bca-b9c9-2199f412a61f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:09.371538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 308/581] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:09.371538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcab18-7128-4bca-b9c9-2199f412a61f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:09.431240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:09.444665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317bdd5-36c2-490e-be55-85b573f969f8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:09.472420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:09.486105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcab18-7128-4bca-b9c9-2199f412a61f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:09.487047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b80c5-96d5-415d-9b53-07c3227f4cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:09.487738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b80c5-96d5-415d-9b53-07c3227f4cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:09.488023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b80c5-96d5-415d-9b53-07c3227f4cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:09.488304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b80c5-96d5-415d-9b53-07c3227f4cd0 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:09.489188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b80c5-96d5-415d-9b53-07c3227f4cd0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:09.489501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b80c5-96d5-415d-9b53-07c3227f4cd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:09.491109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b80c5-96d5-415d-9b53-07c3227f4cd0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:09.525599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 274/582] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 23 11:42:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:09.525599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b80c5-96d5-415d-9b53-07c3227f4cd0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:09.541171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caee1af-f29d-46d5-843d-2bfaf6243d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:09.542062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caee1af-f29d-46d5-843d-2bfaf6243d06 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:09.542415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caee1af-f29d-46d5-843d-2bfaf6243d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:09.542764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caee1af-f29d-46d5-843d-2bfaf6243d06 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:09.552215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caee1af-f29d-46d5-843d-2bfaf6243d06 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:09.554227 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caee1af-f29d-46d5-843d-2bfaf6243d06 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-0caee1af-f29d-46d5-843d-2bfaf6243d06 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:09.573777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:09.575083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:09.576251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:09.576251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:09.576251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:09.576251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:09.576251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:09.576251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:09.576251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:09.576251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:09.576251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 11:42:09.576251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:09.576251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 309/583] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:09.576251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caee1af-f29d-46d5-843d-2bfaf6243d06 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:09.649651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b80c5-96d5-415d-9b53-07c3227f4cd0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:09.690785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caee1af-f29d-46d5-843d-2bfaf6243d06 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:09.691811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f25e2f-1e4a-496c-8a17-c28d92971341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:09.692472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f25e2f-1e4a-496c-8a17-c28d92971341 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:09.692787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f25e2f-1e4a-496c-8a17-c28d92971341 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:09.693779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f25e2f-1e4a-496c-8a17-c28d92971341 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:09.693779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f25e2f-1e4a-496c-8a17-c28d92971341 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:09.694077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f25e2f-1e4a-496c-8a17-c28d92971341 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:09.695447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f25e2f-1e4a-496c-8a17-c28d92971341 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:09.709382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 275/584] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:09.710092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f25e2f-1e4a-496c-8a17-c28d92971341 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:09.724352 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:09.733622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7638325-e13f-48d1-9a25-f986a62b971e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:09.734469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7638325-e13f-48d1-9a25-f986a62b971e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:09.734758 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7638325-e13f-48d1-9a25-f986a62b971e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:09.735038 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7638325-e13f-48d1-9a25-f986a62b971e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:09.746275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7638325-e13f-48d1-9a25-f986a62b971e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:09.747728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7638325-e13f-48d1-9a25-f986a62b971e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:09.758122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-a7638325-e13f-48d1-9a25-f986a62b971e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:09.768618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:09.769641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:09.770531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:09.770531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:09.770531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:09.770531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:09.777669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 310/585] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:09.778595 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7638325-e13f-48d1-9a25-f986a62b971e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:09.825597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f25e2f-1e4a-496c-8a17-c28d92971341 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:09.862351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d7b91-dbbf-4eb3-83b0-23dc7a474b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:09.863136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d7b91-dbbf-4eb3-83b0-23dc7a474b6b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:09.863427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d7b91-dbbf-4eb3-83b0-23dc7a474b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:09.863716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d7b91-dbbf-4eb3-83b0-23dc7a474b6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:09.876174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d7b91-dbbf-4eb3-83b0-23dc7a474b6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:09.877672 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155d7b91-dbbf-4eb3-83b0-23dc7a474b6b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:09.886162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7638325-e13f-48d1-9a25-f986a62b971e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-155d7b91-dbbf-4eb3-83b0-23dc7a474b6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:09.898291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:09.899524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:09.900435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:09.900435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:09.900435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:09.900435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:09.903914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 276/586] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:09.904753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d7b91-dbbf-4eb3-83b0-23dc7a474b6b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:09.924451 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45858a0c-142b-45b1-8899-6bf7b25fced9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:09.925149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45858a0c-142b-45b1-8899-6bf7b25fced9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:09.925448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45858a0c-142b-45b1-8899-6bf7b25fced9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:09.925725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45858a0c-142b-45b1-8899-6bf7b25fced9 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:09.926566 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45858a0c-142b-45b1-8899-6bf7b25fced9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:09.926925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45858a0c-142b-45b1-8899-6bf7b25fced9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:09.928340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45858a0c-142b-45b1-8899-6bf7b25fced9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:09.951281 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 311/587] 10.222.0.67 () {56 vars in 1137 bytes} [Thu Jun 23 11:42:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:09.951281 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45858a0c-142b-45b1-8899-6bf7b25fced9 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:09.976140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:10.011845 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:10.024406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d7b91-dbbf-4eb3-83b0-23dc7a474b6b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:10.060965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45858a0c-142b-45b1-8899-6bf7b25fced9 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:10.061964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fab67c-9eb8-47f9-9cc8-893fad51440d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:10.062767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fab67c-9eb8-47f9-9cc8-893fad51440d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.063057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fab67c-9eb8-47f9-9cc8-893fad51440d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.063345 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fab67c-9eb8-47f9-9cc8-893fad51440d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.070431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fab67c-9eb8-47f9-9cc8-893fad51440d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:10.071881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fab67c-9eb8-47f9-9cc8-893fad51440d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-51fab67c-9eb8-47f9-9cc8-893fad51440d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:10.086153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:10.087275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:10.093141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:10.093141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:10.093141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:10.093141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:10.093141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:10.093141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:10.093141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:10.093141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:10.093141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 277/588] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:10.093141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fab67c-9eb8-47f9-9cc8-893fad51440d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:10.106453 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4c769-14a9-4053-85a4-13272ce26df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:10.107439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4c769-14a9-4053-85a4-13272ce26df2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.107439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4c769-14a9-4053-85a4-13272ce26df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.107701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4c769-14a9-4053-85a4-13272ce26df2 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.108939 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4c769-14a9-4053-85a4-13272ce26df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:10.109300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4c769-14a9-4053-85a4-13272ce26df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:10.110633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4c769-14a9-4053-85a4-13272ce26df2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:10.122022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 312/589] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:10.126558 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4c769-14a9-4053-85a4-13272ce26df2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:10.215081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fab67c-9eb8-47f9-9cc8-893fad51440d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:10.240556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:10.247658 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4c769-14a9-4053-85a4-13272ce26df2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:10.254310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc5be2-2a0b-4e72-8444-13c156646f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:10.254982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc5be2-2a0b-4e72-8444-13c156646f89 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.255668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc5be2-2a0b-4e72-8444-13c156646f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.255968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc5be2-2a0b-4e72-8444-13c156646f89 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.256726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc5be2-2a0b-4e72-8444-13c156646f89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:10.257047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc5be2-2a0b-4e72-8444-13c156646f89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:10.259088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc5be2-2a0b-4e72-8444-13c156646f89 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:10.265509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:10.271896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 278/590] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:10.276801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc5be2-2a0b-4e72-8444-13c156646f89 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:10.284754 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4565ba-67fc-4073-87c8-bfc77e9be93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:10.285522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4565ba-67fc-4073-87c8-bfc77e9be93d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.285831 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4565ba-67fc-4073-87c8-bfc77e9be93d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.286600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4565ba-67fc-4073-87c8-bfc77e9be93d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.299840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4565ba-67fc-4073-87c8-bfc77e9be93d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:10.301618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4565ba-67fc-4073-87c8-bfc77e9be93d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4f4565ba-67fc-4073-87c8-bfc77e9be93d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:10.320127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:10.321506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:10.322411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:10.322411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:10.322411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:10.322411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:10.322411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 313/591] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:10.322411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4565ba-67fc-4073-87c8-bfc77e9be93d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:10.395845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc5be2-2a0b-4e72-8444-13c156646f89 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:10.431202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3798507-fcf0-4273-8a48-e71a762f0a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:10.431859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3798507-fcf0-4273-8a48-e71a762f0a5c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.432147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3798507-fcf0-4273-8a48-e71a762f0a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.432431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3798507-fcf0-4273-8a48-e71a762f0a5c None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.433264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3798507-fcf0-4273-8a48-e71a762f0a5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:10.433583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3798507-fcf0-4273-8a48-e71a762f0a5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:10.433676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4565ba-67fc-4073-87c8-bfc77e9be93d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:10.435014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3798507-fcf0-4273-8a48-e71a762f0a5c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:10.447544 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 279/592] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:10.448365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3798507-fcf0-4273-8a48-e71a762f0a5c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:10.468810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505015c-8fc1-4da2-9267-d9ce5b44a210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:10.469607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505015c-8fc1-4da2-9267-d9ce5b44a210 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.469893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505015c-8fc1-4da2-9267-d9ce5b44a210 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.470175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0505015c-8fc1-4da2-9267-d9ce5b44a210 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.477933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505015c-8fc1-4da2-9267-d9ce5b44a210 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:10.479387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0505015c-8fc1-4da2-9267-d9ce5b44a210 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:10.492841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-0505015c-8fc1-4da2-9267-d9ce5b44a210 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:10.494871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:10.495973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:10.497050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:10.497050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:10.497050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:10.497050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:10.497050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:10.497050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:10.497050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:10.497050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:10.497050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 11:42:10.497050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:10.497050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 314/593] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:10.497050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505015c-8fc1-4da2-9267-d9ce5b44a210 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:10.531578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:10.556042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3798507-fcf0-4273-8a48-e71a762f0a5c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:10.598039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbeefa-ce76-40e0-99a7-b09b20a57510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:10.598744 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbeefa-ce76-40e0-99a7-b09b20a57510 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.599035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbeefa-ce76-40e0-99a7-b09b20a57510 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.599312 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbeefa-ce76-40e0-99a7-b09b20a57510 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.606554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbeefa-ce76-40e0-99a7-b09b20a57510 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:10.608093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbeefa-ce76-40e0-99a7-b09b20a57510 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:10.616558 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0505015c-8fc1-4da2-9267-d9ce5b44a210 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-99fbeefa-ce76-40e0-99a7-b09b20a57510 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:10.622562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:10.623462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:10.624355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:10.624355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:10.624355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:10.624355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:10.624355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:10.624355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:10.624355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:10.624355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:10.624355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 280/594] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:10.625168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de52c13-0a9a-44e4-ad6a-f36aad120a9e None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.625452 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de52c13-0a9a-44e4-ad6a-f36aad120a9e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.625725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de52c13-0a9a-44e4-ad6a-f36aad120a9e None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.626963 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 281/595] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:10.627680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de52c13-0a9a-44e4-ad6a-f36aad120a9e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:10.658324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f76a0c-d185-486f-a203-97d9b028ebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:10.659131 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f76a0c-d185-486f-a203-97d9b028ebec None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.659425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f76a0c-d185-486f-a203-97d9b028ebec None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.659707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f76a0c-d185-486f-a203-97d9b028ebec None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.660455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f76a0c-d185-486f-a203-97d9b028ebec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:10.660796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f76a0c-d185-486f-a203-97d9b028ebec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:10.662205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f76a0c-d185-486f-a203-97d9b028ebec None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:10.676722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 315/596] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:10.677588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f76a0c-d185-486f-a203-97d9b028ebec None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:10.743249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de52c13-0a9a-44e4-ad6a-f36aad120a9e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:10.744828 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:10.771166 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:10.781863 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10cee8-ae50-4888-a421-158c7530d954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:10.783021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10cee8-ae50-4888-a421-158c7530d954 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.783307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10cee8-ae50-4888-a421-158c7530d954 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.783587 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a10cee8-ae50-4888-a421-158c7530d954 None admin] PATH_INFO: `/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/abcbb404d9a148568a82697abd51ec30/roles/ff64135ecc4c47ae978c14600bf31853` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.792373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f76a0c-d185-486f-a203-97d9b028ebec None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:10.799786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10cee8-ae50-4888-a421-158c7530d954 None admin] RBAC: Authorizing `identity:create_grant(project_id=19c2f661cc934aa7869e9b3d4efa875e, user_id=abcbb404d9a148568a82697abd51ec30, role_id=ff64135ecc4c47ae978c14600bf31853)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:10.801299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a10cee8-ae50-4888-a421-158c7530d954 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:10.814494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 282/597] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:42:10 2022] PUT /identity/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/abcbb404d9a148568a82697abd51ec30/roles/ff64135ecc4c47ae978c14600bf31853 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:10.815345 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10cee8-ae50-4888-a421-158c7530d954 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:10.844105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0c6c4-0395-48b2-bfec-51fd6604f9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:10.844819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0c6c4-0395-48b2-bfec-51fd6604f9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.845105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0c6c4-0395-48b2-bfec-51fd6604f9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.845396 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0c6c4-0395-48b2-bfec-51fd6604f9b8 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.846255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0c6c4-0395-48b2-bfec-51fd6604f9b8 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:10.846577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0c6c4-0395-48b2-bfec-51fd6604f9b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:10.848171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0c6c4-0395-48b2-bfec-51fd6604f9b8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:10.879456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 316/598] 10.222.0.67 () {56 vars in 1139 bytes} [Thu Jun 23 11:42:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:10.879456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76235660-a8e9-4c6f-892d-a095e12a8d84 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:10.879456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76235660-a8e9-4c6f-892d-a095e12a8d84 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:10.879456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76235660-a8e9-4c6f-892d-a095e12a8d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:10.896945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-76235660-a8e9-4c6f-892d-a095e12a8d84 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:10.898211 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-76235660-a8e9-4c6f-892d-a095e12a8d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:10.975987 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a10cee8-ae50-4888-a421-158c7530d954 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:10.981506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 317/599] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:10.984596 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76235660-a8e9-4c6f-892d-a095e12a8d84 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:11.022026 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cc784-7472-4eca-aace-66abb426511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:11.022714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cc784-7472-4eca-aace-66abb426511b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:11.023121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cc784-7472-4eca-aace-66abb426511b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:11.023323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cc784-7472-4eca-aace-66abb426511b None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:11.024043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cc784-7472-4eca-aace-66abb426511b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:11.024372 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cc784-7472-4eca-aace-66abb426511b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:11.025938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cc784-7472-4eca-aace-66abb426511b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:11.041502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 283/600] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:11.042420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cc784-7472-4eca-aace-66abb426511b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:11.060521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:11.092977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:11.118012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76235660-a8e9-4c6f-892d-a095e12a8d84 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:11.155879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9d6d6-ee25-4fb1-b68f-ec6af444132c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:11.156905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9d6d6-ee25-4fb1-b68f-ec6af444132c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:11.157452 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9d6d6-ee25-4fb1-b68f-ec6af444132c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:11.157731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9d6d6-ee25-4fb1-b68f-ec6af444132c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:11.165671 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9d6d6-ee25-4fb1-b68f-ec6af444132c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:11.167157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9d6d6-ee25-4fb1-b68f-ec6af444132c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:11.169755 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cc784-7472-4eca-aace-66abb426511b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-07d9d6d6-ee25-4fb1-b68f-ec6af444132c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:11.181399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:11.182443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:11.183508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:11.183508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:11.183508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:11.183508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:11.183508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:11.183508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:11.183508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:11.183508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:11.183508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 318/601] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:11.183508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9d6d6-ee25-4fb1-b68f-ec6af444132c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:11.212958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8fe2e-bf8d-4577-8af9-486af2af0e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:11.213768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8fe2e-bf8d-4577-8af9-486af2af0e1b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:11.214059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8fe2e-bf8d-4577-8af9-486af2af0e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:11.214332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d8fe2e-bf8d-4577-8af9-486af2af0e1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:11.226150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8fe2e-bf8d-4577-8af9-486af2af0e1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:11.227656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d8fe2e-bf8d-4577-8af9-486af2af0e1b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-06d8fe2e-bf8d-4577-8af9-486af2af0e1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:11.249818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:11.250831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:11.251741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:11.251741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:11.251741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:11.251741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:11.251741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 284/602] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:11.252017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8fe2e-bf8d-4577-8af9-486af2af0e1b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:11.318955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9d6d6-ee25-4fb1-b68f-ec6af444132c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:11.328051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:11.360654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af012b5-021b-47e7-a822-98d5736b75ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:11.362233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af012b5-021b-47e7-a822-98d5736b75ea None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:11.362484 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af012b5-021b-47e7-a822-98d5736b75ea None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:11.362859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af012b5-021b-47e7-a822-98d5736b75ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:11.370927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:11.375869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d8fe2e-bf8d-4577-8af9-486af2af0e1b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:11.377374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af012b5-021b-47e7-a822-98d5736b75ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-5af012b5-021b-47e7-a822-98d5736b75ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:11.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:11.390893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 319/603] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:11.398311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af012b5-021b-47e7-a822-98d5736b75ea None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:11.417490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb1692-7c6f-42bb-acc8-ebfc58b2a87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:11.418786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb1692-7c6f-42bb-acc8-ebfc58b2a87f None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:11.419256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb1692-7c6f-42bb-acc8-ebfc58b2a87f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:11.419727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb1692-7c6f-42bb-acc8-ebfc58b2a87f None admin] PATH_INFO: `/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/36c18bbd22f74efbae7aab1cdd3f6403/roles/ff64135ecc4c47ae978c14600bf31853` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:11.444957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb1692-7c6f-42bb-acc8-ebfc58b2a87f None admin] RBAC: Authorizing `identity:create_grant(project_id=19c2f661cc934aa7869e9b3d4efa875e, user_id=36c18bbd22f74efbae7aab1cdd3f6403, role_id=ff64135ecc4c47ae978c14600bf31853)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:11.446804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb1692-7c6f-42bb-acc8-ebfc58b2a87f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:11.472727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 285/604] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:42:11 2022] PUT /identity/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/36c18bbd22f74efbae7aab1cdd3f6403/roles/ff64135ecc4c47ae978c14600bf31853 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:11.472727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb1692-7c6f-42bb-acc8-ebfc58b2a87f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:11.528108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af012b5-021b-47e7-a822-98d5736b75ea None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:11.582997 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a988d7-37d4-40d4-b126-fcae9f745986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:11.583687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a988d7-37d4-40d4-b126-fcae9f745986 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:11.583979 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a988d7-37d4-40d4-b126-fcae9f745986 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:11.584264 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a988d7-37d4-40d4-b126-fcae9f745986 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:11.585419 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a988d7-37d4-40d4-b126-fcae9f745986 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:11.585750 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a988d7-37d4-40d4-b126-fcae9f745986 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:11.587248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a988d7-37d4-40d4-b126-fcae9f745986 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:11.602658 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 320/605] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:11.603585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a988d7-37d4-40d4-b126-fcae9f745986 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:11.641046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb1692-7c6f-42bb-acc8-ebfc58b2a87f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:11.667432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:11.684518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:11.694352 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c53c5-f1e5-4ebd-992c-6b2cc2b148c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:11.694352 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c53c5-f1e5-4ebd-992c-6b2cc2b148c5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:11.694882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c53c5-f1e5-4ebd-992c-6b2cc2b148c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:11.694882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c53c5-f1e5-4ebd-992c-6b2cc2b148c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:11.696037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c53c5-f1e5-4ebd-992c-6b2cc2b148c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:11.696351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c53c5-f1e5-4ebd-992c-6b2cc2b148c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:11.698340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c53c5-f1e5-4ebd-992c-6b2cc2b148c5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:11.711942 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 286/606] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:11.712909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c53c5-f1e5-4ebd-992c-6b2cc2b148c5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:11.752391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a988d7-37d4-40d4-b126-fcae9f745986 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:11.799032 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746e583-5bf8-4d5d-93a2-5ad098c98e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:11.800359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746e583-5bf8-4d5d-93a2-5ad098c98e52 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:11.801291 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746e583-5bf8-4d5d-93a2-5ad098c98e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:11.802053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746e583-5bf8-4d5d-93a2-5ad098c98e52 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:11.803026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746e583-5bf8-4d5d-93a2-5ad098c98e52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:11.803545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746e583-5bf8-4d5d-93a2-5ad098c98e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:11.806138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8746e583-5bf8-4d5d-93a2-5ad098c98e52 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:11.819146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 321/607] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:11.820373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746e583-5bf8-4d5d-93a2-5ad098c98e52 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:11.854535 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c53c5-f1e5-4ebd-992c-6b2cc2b148c5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:11.894333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda00dd7-a169-4a09-adc2-c9521a58ace4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:11.895680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda00dd7-a169-4a09-adc2-c9521a58ace4 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:11.896296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda00dd7-a169-4a09-adc2-c9521a58ace4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:11.896932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda00dd7-a169-4a09-adc2-c9521a58ace4 None admin] PATH_INFO: `/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/74fdc12d728d45018ea668a4073da3b8/roles/ff64135ecc4c47ae978c14600bf31853` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:11.914627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda00dd7-a169-4a09-adc2-c9521a58ace4 None admin] RBAC: Authorizing `identity:create_grant(project_id=19c2f661cc934aa7869e9b3d4efa875e, user_id=74fdc12d728d45018ea668a4073da3b8, role_id=ff64135ecc4c47ae978c14600bf31853)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:11.916550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda00dd7-a169-4a09-adc2-c9521a58ace4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:11.936224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 287/608] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:42:11 2022] PUT /identity/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/74fdc12d728d45018ea668a4073da3b8/roles/ff64135ecc4c47ae978c14600bf31853 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:11.937075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda00dd7-a169-4a09-adc2-c9521a58ace4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:11.953384 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746e583-5bf8-4d5d-93a2-5ad098c98e52 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:11.995266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:12.005733 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:12.037316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02fa53-49e3-42ef-813b-f3e63f666c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:12.039626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02fa53-49e3-42ef-813b-f3e63f666c88 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:12.039915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02fa53-49e3-42ef-813b-f3e63f666c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:12.040197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02fa53-49e3-42ef-813b-f3e63f666c88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:12.056951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02fa53-49e3-42ef-813b-f3e63f666c88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:12.058903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02fa53-49e3-42ef-813b-f3e63f666c88 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-1f02fa53-49e3-42ef-813b-f3e63f666c88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:12.082124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:12.083156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:12.084276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:12.084276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:12.084276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:12.084276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:12.084276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:12.084276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:12.084276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:12.084276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:12.084276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 322/609] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:12.090589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02fa53-49e3-42ef-813b-f3e63f666c88 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:12.179854 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda00dd7-a169-4a09-adc2-c9521a58ace4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:12.216244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02fa53-49e3-42ef-813b-f3e63f666c88 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:12.244726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c24b3f-3245-4e82-a934-883ef420b42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:12.245528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c24b3f-3245-4e82-a934-883ef420b42c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:12.245812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c24b3f-3245-4e82-a934-883ef420b42c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:12.246086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c24b3f-3245-4e82-a934-883ef420b42c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:12.256013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8e1f6-1275-42ea-b1e4-17e26a101884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:12.256709 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8e1f6-1275-42ea-b1e4-17e26a101884 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:12.257000 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8e1f6-1275-42ea-b1e4-17e26a101884 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:12.257281 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8e1f6-1275-42ea-b1e4-17e26a101884 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:12.257757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c24b3f-3245-4e82-a934-883ef420b42c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:12.258086 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8e1f6-1275-42ea-b1e4-17e26a101884 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:12.258411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8e1f6-1275-42ea-b1e4-17e26a101884 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:12.259214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c24b3f-3245-4e82-a934-883ef420b42c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:12.260202 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8e1f6-1275-42ea-b1e4-17e26a101884 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:12.273554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 323/610] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-e5c24b3f-3245-4e82-a934-883ef420b42c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:12.277698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:12.279768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:12.282007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:12.282007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:12.282007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:12.282007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:12.282007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 288/611] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:12.282908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8e1f6-1275-42ea-b1e4-17e26a101884 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:12.288697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c24b3f-3245-4e82-a934-883ef420b42c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:12.360013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:12.380108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:12.427843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c24b3f-3245-4e82-a934-883ef420b42c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:12.484402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8e1f6-1275-42ea-b1e4-17e26a101884 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:12.501767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad8d5f-f5ba-471f-a7c0-d34c073ab766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:12.502445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad8d5f-f5ba-471f-a7c0-d34c073ab766 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:12.502780 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad8d5f-f5ba-471f-a7c0-d34c073ab766 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:12.503057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad8d5f-f5ba-471f-a7c0-d34c073ab766 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:12.504053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad8d5f-f5ba-471f-a7c0-d34c073ab766 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:12.504365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad8d5f-f5ba-471f-a7c0-d34c073ab766 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:12.505800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad8d5f-f5ba-471f-a7c0-d34c073ab766 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:12.532883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 289/612] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:12.535479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad8d5f-f5ba-471f-a7c0-d34c073ab766 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:12.565274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aad983-fb76-4b32-9617-0d6c1e6ae8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:12.566167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aad983-fb76-4b32-9617-0d6c1e6ae8a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:12.566447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aad983-fb76-4b32-9617-0d6c1e6ae8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:12.566722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aad983-fb76-4b32-9617-0d6c1e6ae8a0 None admin] PATH_INFO: `/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/5f095d3370be457cb6687ac8f8c9f8ab/roles/ff64135ecc4c47ae978c14600bf31853` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:12.598367 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aad983-fb76-4b32-9617-0d6c1e6ae8a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=19c2f661cc934aa7869e9b3d4efa875e, user_id=5f095d3370be457cb6687ac8f8c9f8ab, role_id=ff64135ecc4c47ae978c14600bf31853)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:12.599864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aad983-fb76-4b32-9617-0d6c1e6ae8a0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:12.629792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 324/613] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:42:12 2022] PUT /identity/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/5f095d3370be457cb6687ac8f8c9f8ab/roles/ff64135ecc4c47ae978c14600bf31853 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:12.707093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad8d5f-f5ba-471f-a7c0-d34c073ab766 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:12.762303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80738460-1996-40ca-bab4-66e8b93c1a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:12.763694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80738460-1996-40ca-bab4-66e8b93c1a76 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:12.764122 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80738460-1996-40ca-bab4-66e8b93c1a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:12.764497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80738460-1996-40ca-bab4-66e8b93c1a76 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:12.777056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80738460-1996-40ca-bab4-66e8b93c1a76 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:12.778685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80738460-1996-40ca-bab4-66e8b93c1a76 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:12.789112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-80738460-1996-40ca-bab4-66e8b93c1a76 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:12.804357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:12.806943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:12.809360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:12.809360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:12.809360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:12.809360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:12.809360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:12.809360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:12.809360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:12.809360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:12.809360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:42:12.809360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:12.809360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 290/614] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:42:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:12.814135 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aad983-fb76-4b32-9617-0d6c1e6ae8a0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:13.037317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:13.069577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aad983-fb76-4b32-9617-0d6c1e6ae8a0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:13.148750 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db3846-c2f3-4124-86b6-f2f2558810db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:13.149429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db3846-c2f3-4124-86b6-f2f2558810db None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:13.149699 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db3846-c2f3-4124-86b6-f2f2558810db None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:13.150043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db3846-c2f3-4124-86b6-f2f2558810db None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:13.150950 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db3846-c2f3-4124-86b6-f2f2558810db None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:13.151261 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db3846-c2f3-4124-86b6-f2f2558810db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:13.152758 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3db3846-c2f3-4124-86b6-f2f2558810db None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:13.158643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427682fd-de55-4779-9975-03a0216bd783 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:13.160104 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427682fd-de55-4779-9975-03a0216bd783 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:13.160918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427682fd-de55-4779-9975-03a0216bd783 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:13.168828 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 291/615] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:13.173011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884feab-a459-4b15-b8e2-c61dd08248b6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:13.173347 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884feab-a459-4b15-b8e2-c61dd08248b6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:13.173597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884feab-a459-4b15-b8e2-c61dd08248b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:13.190875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 325/616] 10.222.0.67 () {56 vars in 1135 bytes} [Thu Jun 23 11:42:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:13.197152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db3846-c2f3-4124-86b6-f2f2558810db None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:13.215518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-4884feab-a459-4b15-b8e2-c61dd08248b6 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:13.219023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4884feab-a459-4b15-b8e2-c61dd08248b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:13.324585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db3846-c2f3-4124-86b6-f2f2558810db None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:13.369266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609711a-5e5b-4977-b127-5ab7216441b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:13.370788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609711a-5e5b-4977-b127-5ab7216441b7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:13.372804 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609711a-5e5b-4977-b127-5ab7216441b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:13.373472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609711a-5e5b-4977-b127-5ab7216441b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:13.395987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609711a-5e5b-4977-b127-5ab7216441b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:13.398640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609711a-5e5b-4977-b127-5ab7216441b7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:13.413447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-5609711a-5e5b-4977-b127-5ab7216441b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:13.422425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:13.423447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:13.424866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:13.424866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:13.424866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:13.424866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:13.424866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 326/617] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:13.430125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609711a-5e5b-4977-b127-5ab7216441b7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:13.436789 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 292/618] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:42:13.456637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884feab-a459-4b15-b8e2-c61dd08248b6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:13.548763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609711a-5e5b-4977-b127-5ab7216441b7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:13.659042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884feab-a459-4b15-b8e2-c61dd08248b6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:13.673432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b737bb-f9cd-4faa-8526-587e72d7747f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:13.674131 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b737bb-f9cd-4faa-8526-587e72d7747f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:13.674426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b737bb-f9cd-4faa-8526-587e72d7747f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:13.674715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b737bb-f9cd-4faa-8526-587e72d7747f None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:13.675466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b737bb-f9cd-4faa-8526-587e72d7747f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:13.675786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b737bb-f9cd-4faa-8526-587e72d7747f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:13.677254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b737bb-f9cd-4faa-8526-587e72d7747f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:13.690316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:13.703429 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b7585-cbe1-4984-aee3-6a0de567a922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:13.704906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b7585-cbe1-4984-aee3-6a0de567a922 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:13.705385 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b7585-cbe1-4984-aee3-6a0de567a922 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:13.705834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b7585-cbe1-4984-aee3-6a0de567a922 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:13.707574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 327/619] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:13.719795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b737bb-f9cd-4faa-8526-587e72d7747f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:13.727828 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b7585-cbe1-4984-aee3-6a0de567a922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f02b7585-cbe1-4984-aee3-6a0de567a922 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:13.747137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:13.749773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:13.751349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:13.751349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:13.751349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:13.751349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:13.751349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:13.751349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:13.751349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:13.751349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:13.751349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:13.751349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:13.751349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:13.751349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:13.758995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 293/620] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:13.764820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b7585-cbe1-4984-aee3-6a0de567a922 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:13.844416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:13.893303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b737bb-f9cd-4faa-8526-587e72d7747f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:13.934729 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ad716-878f-4fe0-a159-ac5195d29f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:13.935917 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ad716-878f-4fe0-a159-ac5195d29f43 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:13.936965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ad716-878f-4fe0-a159-ac5195d29f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:13.937404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ad716-878f-4fe0-a159-ac5195d29f43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:13.946540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ad716-878f-4fe0-a159-ac5195d29f43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:13.948836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ad716-878f-4fe0-a159-ac5195d29f43 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-490ad716-878f-4fe0-a159-ac5195d29f43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:13.964311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:13.965526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:13.966605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:13.966605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:13.966605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:13.966605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:13.966605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:13.966605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:13.966605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:13.966605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:13.969215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 328/621] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:13.974122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a4070-1f9b-4533-92ba-8cdff9d9ed90 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:13.974529 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a4070-1f9b-4533-92ba-8cdff9d9ed90 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:13.974884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a4070-1f9b-4533-92ba-8cdff9d9ed90 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:13.976391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 329/622] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:42:13.977391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a4070-1f9b-4533-92ba-8cdff9d9ed90 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:13.997002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b7585-cbe1-4984-aee3-6a0de567a922 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:14.033579 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55518a39-2677-4438-914e-e518dbc776ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:14.034253 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55518a39-2677-4438-914e-e518dbc776ef None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:14.034539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55518a39-2677-4438-914e-e518dbc776ef None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:14.034818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55518a39-2677-4438-914e-e518dbc776ef None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:14.035734 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55518a39-2677-4438-914e-e518dbc776ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:14.036073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55518a39-2677-4438-914e-e518dbc776ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:14.037499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55518a39-2677-4438-914e-e518dbc776ef None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:14.047109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 294/623] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:14.051956 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f55e26-5baa-4fe7-a61b-a3679d33c971 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:14.052227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f55e26-5baa-4fe7-a61b-a3679d33c971 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:14.052514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f55e26-5baa-4fe7-a61b-a3679d33c971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:14.072292 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f5f55e26-5baa-4fe7-a61b-a3679d33c971 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:14.073937 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f5f55e26-5baa-4fe7-a61b-a3679d33c971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:14.110850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a4070-1f9b-4533-92ba-8cdff9d9ed90 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:14.133099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:14.139076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:14.150013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 295/624] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:14.151463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f55e26-5baa-4fe7-a61b-a3679d33c971 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:14.158078 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e849e-0253-46c8-a95c-5ddf559a4ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:14.158974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e849e-0253-46c8-a95c-5ddf559a4ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:14.159445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e849e-0253-46c8-a95c-5ddf559a4ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:14.159923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e849e-0253-46c8-a95c-5ddf559a4ed2 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:14.161321 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e849e-0253-46c8-a95c-5ddf559a4ed2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:14.161845 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e849e-0253-46c8-a95c-5ddf559a4ed2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:14.163469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e849e-0253-46c8-a95c-5ddf559a4ed2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:14.177565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 330/625] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:14.183243 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e849e-0253-46c8-a95c-5ddf559a4ed2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:14.283296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f55e26-5baa-4fe7-a61b-a3679d33c971 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:14.302696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e849e-0253-46c8-a95c-5ddf559a4ed2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:14.323184 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5cdb0-51e9-41ee-9696-d25219dc7a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:14.324252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5cdb0-51e9-41ee-9696-d25219dc7a63 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:14.324738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5cdb0-51e9-41ee-9696-d25219dc7a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:14.325249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5cdb0-51e9-41ee-9696-d25219dc7a63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:14.336970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5cdb0-51e9-41ee-9696-d25219dc7a63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:14.338790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5cdb0-51e9-41ee-9696-d25219dc7a63 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:14.340405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff9a0c-760e-42f4-9cfd-e1723e4a5bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:14.341376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff9a0c-760e-42f4-9cfd-e1723e4a5bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:14.341666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff9a0c-760e-42f4-9cfd-e1723e4a5bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:14.341948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff9a0c-760e-42f4-9cfd-e1723e4a5bf7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:14.352994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeff9a0c-760e-42f4-9cfd-e1723e4a5bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f5d5cdb0-51e9-41ee-9696-d25219dc7a63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:14.358928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:14.359826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:14.360761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:14.360761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:14.360761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:14.360761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-eeff9a0c-760e-42f4-9cfd-e1723e4a5bf7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:14.361474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:14.362366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:14.363201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:14.363201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:14.363201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:14.363201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:14.363201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:14.363201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:14.363201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:14.363201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:14.363201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:14.363201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:14.363201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:14.363201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:14.365844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 296/627] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:14.366550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 331/627] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:14.367093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5cdb0-51e9-41ee-9696-d25219dc7a63 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:14.369028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff9a0c-760e-42f4-9cfd-e1723e4a5bf7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:14.401486 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:14.418125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:14.477863 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5cdb0-51e9-41ee-9696-d25219dc7a63 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:14.481513 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff9a0c-760e-42f4-9cfd-e1723e4a5bf7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:14.516985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5eeb4-e09f-42c0-9066-984a7befe744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:14.518321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5eeb4-e09f-42c0-9066-984a7befe744 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:14.518724 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52f21d-8778-4e02-aaeb-986d8ad5f826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:14.519098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5eeb4-e09f-42c0-9066-984a7befe744 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:14.519394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52f21d-8778-4e02-aaeb-986d8ad5f826 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:14.519678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52f21d-8778-4e02-aaeb-986d8ad5f826 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:14.519955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52f21d-8778-4e02-aaeb-986d8ad5f826 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:14.520231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5eeb4-e09f-42c0-9066-984a7befe744 None admin] PATH_INFO: `/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/eb9b9af5e0b44e34837e1cdb5ff6d33e/roles/ff64135ecc4c47ae978c14600bf31853` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:14.520749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52f21d-8778-4e02-aaeb-986d8ad5f826 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:14.521066 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52f21d-8778-4e02-aaeb-986d8ad5f826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:14.522593 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b52f21d-8778-4e02-aaeb-986d8ad5f826 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:14.536875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 332/628] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:14.537824 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52f21d-8778-4e02-aaeb-986d8ad5f826 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:14.539196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5eeb4-e09f-42c0-9066-984a7befe744 None admin] RBAC: Authorizing `identity:create_grant(project_id=19c2f661cc934aa7869e9b3d4efa875e, user_id=eb9b9af5e0b44e34837e1cdb5ff6d33e, role_id=ff64135ecc4c47ae978c14600bf31853)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:14.541002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5eeb4-e09f-42c0-9066-984a7befe744 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:14.556431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 297/629] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:42:14 2022] PUT /identity/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/eb9b9af5e0b44e34837e1cdb5ff6d33e/roles/ff64135ecc4c47ae978c14600bf31853 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:14.557629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5eeb4-e09f-42c0-9066-984a7befe744 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:14.655580 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:14.679653 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b52f21d-8778-4e02-aaeb-986d8ad5f826 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:14.680573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5eeb4-e09f-42c0-9066-984a7befe744 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:14.682371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:14.719906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c491dd-25c8-4982-85c7-2f988448450d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:14.720788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c491dd-25c8-4982-85c7-2f988448450d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:14.721075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c491dd-25c8-4982-85c7-2f988448450d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:14.721349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c491dd-25c8-4982-85c7-2f988448450d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:14.724509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433a28b-f4a0-4bb4-9d7e-918d0abed95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:14.725699 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433a28b-f4a0-4bb4-9d7e-918d0abed95f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:14.726144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433a28b-f4a0-4bb4-9d7e-918d0abed95f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:14.726601 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433a28b-f4a0-4bb4-9d7e-918d0abed95f None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:14.727485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433a28b-f4a0-4bb4-9d7e-918d0abed95f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:14.727958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433a28b-f4a0-4bb4-9d7e-918d0abed95f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:14.730058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433a28b-f4a0-4bb4-9d7e-918d0abed95f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:14.732547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c491dd-25c8-4982-85c7-2f988448450d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:14.734832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c491dd-25c8-4982-85c7-2f988448450d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:14.746055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 333/630] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:14.747213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fef258-cf73-4612-ac0e-f359be4f85f7 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:14.747578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fef258-cf73-4612-ac0e-f359be4f85f7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:14.750644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fef258-cf73-4612-ac0e-f359be4f85f7 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:14.750644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 334/631] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:14.753029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fef258-cf73-4612-ac0e-f359be4f85f7 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-c5c491dd-25c8-4982-85c7-2f988448450d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:14.758503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:14.759428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:14.761089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:14.761089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:14.761089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:14.761089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:14.761089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 298/632] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:14.761422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4d36d-8e39-4a6d-99ed-0a8bb35faf06 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:14.761422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4d36d-8e39-4a6d-99ed-0a8bb35faf06 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:14.762644 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4d36d-8e39-4a6d-99ed-0a8bb35faf06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:14.778443 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0be4d36d-8e39-4a6d-99ed-0a8bb35faf06 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:14.779639 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0be4d36d-8e39-4a6d-99ed-0a8bb35faf06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:14.857264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 299/633] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:14.858692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4d36d-8e39-4a6d-99ed-0a8bb35faf06 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:14.894331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fef258-cf73-4612-ac0e-f359be4f85f7 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:14.927716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:14.939137 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e83e05-27a1-4355-8e03-c7a3ca559e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:14.940297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e83e05-27a1-4355-8e03-c7a3ca559e9b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:14.940824 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e83e05-27a1-4355-8e03-c7a3ca559e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:14.941374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e83e05-27a1-4355-8e03-c7a3ca559e9b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:14.949440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e83e05-27a1-4355-8e03-c7a3ca559e9b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:14.951236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e83e05-27a1-4355-8e03-c7a3ca559e9b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-92e83e05-27a1-4355-8e03-c7a3ca559e9b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:14.967994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:14.969016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:14.969987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:14.969987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:14.969987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:14.969987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:14.969987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:14.969987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:14.969987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:14.969987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:14.969987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:42:14.969987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:14.970532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4d36d-8e39-4a6d-99ed-0a8bb35faf06 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:14.972908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 335/634] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:14.973988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e83e05-27a1-4355-8e03-c7a3ca559e9b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:15.001120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:15.006321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e759c0c-414f-45bb-be08-27cd2aea4c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:15.006992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e759c0c-414f-45bb-be08-27cd2aea4c3d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:15.007282 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e759c0c-414f-45bb-be08-27cd2aea4c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:15.007565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e759c0c-414f-45bb-be08-27cd2aea4c3d None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:15.008471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e759c0c-414f-45bb-be08-27cd2aea4c3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:15.008666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e759c0c-414f-45bb-be08-27cd2aea4c3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:15.010562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e759c0c-414f-45bb-be08-27cd2aea4c3d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:15.023414 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 300/635] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:15.024313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e759c0c-414f-45bb-be08-27cd2aea4c3d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:15.084791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e83e05-27a1-4355-8e03-c7a3ca559e9b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:15.121810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb35f51-0ba1-44c3-a319-be30edf72cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:15.123107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb35f51-0ba1-44c3-a319-be30edf72cba None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:15.123595 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb35f51-0ba1-44c3-a319-be30edf72cba None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:15.124031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb35f51-0ba1-44c3-a319-be30edf72cba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:15.135774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb35f51-0ba1-44c3-a319-be30edf72cba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:15.140209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e759c0c-414f-45bb-be08-27cd2aea4c3d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-9cb35f51-0ba1-44c3-a319-be30edf72cba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:15.145060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:15.146014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:15.146877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:15.146877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:15.146877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:15.146877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:15.146877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:15.146877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:15.146877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:15.146877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:15.146877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:15.146877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:15.146877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:15.146877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:15.153900 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 336/636] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:15.155630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb35f51-0ba1-44c3-a319-be30edf72cba None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:15.169499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:15.183388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912cc58-1f83-4915-86fa-4cbd7684360f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:15.184174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912cc58-1f83-4915-86fa-4cbd7684360f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:15.184466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912cc58-1f83-4915-86fa-4cbd7684360f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:15.184789 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912cc58-1f83-4915-86fa-4cbd7684360f None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:15.185867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912cc58-1f83-4915-86fa-4cbd7684360f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:15.186082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912cc58-1f83-4915-86fa-4cbd7684360f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:15.187598 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912cc58-1f83-4915-86fa-4cbd7684360f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:15.212034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 301/637] 10.222.0.67 () {56 vars in 1137 bytes} [Thu Jun 23 11:42:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:15.212979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912cc58-1f83-4915-86fa-4cbd7684360f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:15.251109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:15.270871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb35f51-0ba1-44c3-a319-be30edf72cba None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:15.305954 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729864d-9f68-4ab5-a9ec-f768c847593e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:15.306744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729864d-9f68-4ab5-a9ec-f768c847593e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:15.307029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729864d-9f68-4ab5-a9ec-f768c847593e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:15.307511 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729864d-9f68-4ab5-a9ec-f768c847593e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:15.315580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729864d-9f68-4ab5-a9ec-f768c847593e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:15.316998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729864d-9f68-4ab5-a9ec-f768c847593e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:15.326343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912cc58-1f83-4915-86fa-4cbd7684360f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2729864d-9f68-4ab5-a9ec-f768c847593e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:15.331853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:15.332830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:15.333868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:15.333868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:15.333868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:15.333868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:15.333868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:15.333868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:15.333868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:15.333868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:15.333868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:42:15.333868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:15.333868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 337/638] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 23 11:42:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:15.333868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729864d-9f68-4ab5-a9ec-f768c847593e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:15.369575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a311702-4d42-4279-8774-d0537fe502df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:15.370250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a311702-4d42-4279-8774-d0537fe502df None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:15.370537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a311702-4d42-4279-8774-d0537fe502df None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:15.370815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a311702-4d42-4279-8774-d0537fe502df None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:15.371573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a311702-4d42-4279-8774-d0537fe502df None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:15.371935 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a311702-4d42-4279-8774-d0537fe502df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:15.373599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a311702-4d42-4279-8774-d0537fe502df None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:15.382838 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 302/639] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:15.384225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859fe51-c833-4ff9-b6bb-d8c15497fec8 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:15.384518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859fe51-c833-4ff9-b6bb-d8c15497fec8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:15.384921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859fe51-c833-4ff9-b6bb-d8c15497fec8 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:15.386344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 303/640] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:15.390966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859fe51-c833-4ff9-b6bb-d8c15497fec8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:15.439244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:15.450386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729864d-9f68-4ab5-a9ec-f768c847593e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:15.483972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8175d99-9020-41b0-a667-9d7baf9f03a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:15.484646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8175d99-9020-41b0-a667-9d7baf9f03a6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:15.484959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8175d99-9020-41b0-a667-9d7baf9f03a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:15.485237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8175d99-9020-41b0-a667-9d7baf9f03a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:15.502164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:15.506784 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859fe51-c833-4ff9-b6bb-d8c15497fec8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:15.541308 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c417d-7feb-4601-8809-7e64bbaec9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:15.541982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c417d-7feb-4601-8809-7e64bbaec9dc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:15.542272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c417d-7feb-4601-8809-7e64bbaec9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:15.542550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c417d-7feb-4601-8809-7e64bbaec9dc None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:15.543392 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c417d-7feb-4601-8809-7e64bbaec9dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:15.543705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c417d-7feb-4601-8809-7e64bbaec9dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:15.545176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c417d-7feb-4601-8809-7e64bbaec9dc None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:15.564882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 304/641] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 23 11:42:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:15.565789 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c417d-7feb-4601-8809-7e64bbaec9dc None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:15.575811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8175d99-9020-41b0-a667-9d7baf9f03a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:15.577501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8175d99-9020-41b0-a667-9d7baf9f03a6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:15.672380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:15.676364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c417d-7feb-4601-8809-7e64bbaec9dc None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:15.686455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 338/642] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:42:15.687770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d4b2-dafc-43af-bca1-a3266ba2b4d2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:15.688059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d4b2-dafc-43af-bca1-a3266ba2b4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:15.688336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60d4b2-dafc-43af-bca1-a3266ba2b4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:15.704465 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-af60d4b2-dafc-43af-bca1-a3266ba2b4d2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:15.705782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-af60d4b2-dafc-43af-bca1-a3266ba2b4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:15.716793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab16f2-000e-4446-99dd-a7d6cef99f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:15.718170 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab16f2-000e-4446-99dd-a7d6cef99f49 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:15.718461 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab16f2-000e-4446-99dd-a7d6cef99f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:15.718736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab16f2-000e-4446-99dd-a7d6cef99f49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:15.729235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab16f2-000e-4446-99dd-a7d6cef99f49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:15.730688 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab16f2-000e-4446-99dd-a7d6cef99f49 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:15.743300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d7ab16f2-000e-4446-99dd-a7d6cef99f49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:15.749704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:15.750794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:15.751714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:15.751714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:15.751714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:15.751714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:15.751714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 305/643] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:15.752021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab16f2-000e-4446-99dd-a7d6cef99f49 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:15.783679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 339/644] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:15.784455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60d4b2-dafc-43af-bca1-a3266ba2b4d2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:15.881359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab16f2-000e-4446-99dd-a7d6cef99f49 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:15.903817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60d4b2-dafc-43af-bca1-a3266ba2b4d2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:15.919692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6f6ca-c518-4ce0-aee8-59aabc951869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:15.920271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6f6ca-c518-4ce0-aee8-59aabc951869 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:15.921128 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6f6ca-c518-4ce0-aee8-59aabc951869 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:15.921410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6f6ca-c518-4ce0-aee8-59aabc951869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:15.942458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb33890-200d-4639-8862-7b467c5f3f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:15.943237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb33890-200d-4639-8862-7b467c5f3f5c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:15.943517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb33890-200d-4639-8862-7b467c5f3f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:15.944062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb33890-200d-4639-8862-7b467c5f3f5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:15.945380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:15.955546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb33890-200d-4639-8862-7b467c5f3f5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:15.957011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb33890-200d-4639-8862-7b467c5f3f5c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4eb33890-200d-4639-8862-7b467c5f3f5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:15.976170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:15.977177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:15.978256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:15.978256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:15.978256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:15.978256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:15.978256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 340/645] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:15.978256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb33890-200d-4639-8862-7b467c5f3f5c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:16.001860 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6f6ca-c518-4ce0-aee8-59aabc951869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:16.003319 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6f6ca-c518-4ce0-aee8-59aabc951869 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:16.007786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:16.088771 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb33890-200d-4639-8862-7b467c5f3f5c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:16.113833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 306/646] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:42:16.113833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6f6ca-c518-4ce0-aee8-59aabc951869 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:16.130003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaf228-8231-4d4f-9088-f98899afce3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:16.130671 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaf228-8231-4d4f-9088-f98899afce3b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.130961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaf228-8231-4d4f-9088-f98899afce3b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.131257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaf228-8231-4d4f-9088-f98899afce3b None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.131980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaf228-8231-4d4f-9088-f98899afce3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:16.132290 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaf228-8231-4d4f-9088-f98899afce3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:16.133712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aaf228-8231-4d4f-9088-f98899afce3b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:16.148709 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 341/647] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:16.149570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaf228-8231-4d4f-9088-f98899afce3b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:16.173138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:16.225782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6f6ca-c518-4ce0-aee8-59aabc951869 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:16.248865 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:16.268637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75b577-7559-4de8-a58a-55bae5fb4c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:16.269476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75b577-7559-4de8-a58a-55bae5fb4c39 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.269767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75b577-7559-4de8-a58a-55bae5fb4c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.270044 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d75b577-7559-4de8-a58a-55bae5fb4c39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.273449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaf228-8231-4d4f-9088-f98899afce3b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:16.287967 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d75b577-7559-4de8-a58a-55bae5fb4c39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-1d75b577-7559-4de8-a58a-55bae5fb4c39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:16.296262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:16.297255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:16.298113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 307/648] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:16.299387 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75b577-7559-4de8-a58a-55bae5fb4c39 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:16.318934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd02f4-6879-4b7e-b27c-ae910ae608ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:16.319933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd02f4-6879-4b7e-b27c-ae910ae608ba None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.320384 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd02f4-6879-4b7e-b27c-ae910ae608ba None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.320875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd02f4-6879-4b7e-b27c-ae910ae608ba None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.321852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd02f4-6879-4b7e-b27c-ae910ae608ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:16.322323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd02f4-6879-4b7e-b27c-ae910ae608ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:16.324013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd02f4-6879-4b7e-b27c-ae910ae608ba None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:16.341700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 342/649] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:16.342885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd02f4-6879-4b7e-b27c-ae910ae608ba None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:16.429456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d75b577-7559-4de8-a58a-55bae5fb4c39 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:16.443675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:16.461386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd02f4-6879-4b7e-b27c-ae910ae608ba None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:16.466106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b33e87-af96-4df7-ab02-a4e2e5d8450e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:16.466902 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b33e87-af96-4df7-ab02-a4e2e5d8450e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.467192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b33e87-af96-4df7-ab02-a4e2e5d8450e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.467498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b33e87-af96-4df7-ab02-a4e2e5d8450e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.478079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b33e87-af96-4df7-ab02-a4e2e5d8450e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:16.479536 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b33e87-af96-4df7-ab02-a4e2e5d8450e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:16.496883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58be62-b77c-41b8-93ad-bb99814e2e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-88b33e87-af96-4df7-ab02-a4e2e5d8450e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:16.497728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:16.499139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:16.500018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:16.500018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:16.500018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:16.500018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:16.500248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58be62-b77c-41b8-93ad-bb99814e2e8c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.500248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58be62-b77c-41b8-93ad-bb99814e2e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.500248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58be62-b77c-41b8-93ad-bb99814e2e8c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.501205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 308/650] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:16.501205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b33e87-af96-4df7-ab02-a4e2e5d8450e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:16.509538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58be62-b77c-41b8-93ad-bb99814e2e8c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:16.511042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58be62-b77c-41b8-93ad-bb99814e2e8c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-fc58be62-b77c-41b8-93ad-bb99814e2e8c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:16.526951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:16.527923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:16.528911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:16.528911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:16.528911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:16.528911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:16.528911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:16.528911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:16.528911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:16.528911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:16.528911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 11:42:16.528911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:16.528911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 343/651] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:16.530212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58be62-b77c-41b8-93ad-bb99814e2e8c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:16.533455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:16.609783 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b33e87-af96-4df7-ab02-a4e2e5d8450e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:16.638265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58be62-b77c-41b8-93ad-bb99814e2e8c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:16.646849 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69677bc-9320-495a-bbde-c541e6c490e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:16.647532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69677bc-9320-495a-bbde-c541e6c490e0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.647820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69677bc-9320-495a-bbde-c541e6c490e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.648111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69677bc-9320-495a-bbde-c541e6c490e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.648900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69677bc-9320-495a-bbde-c541e6c490e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:16.649215 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69677bc-9320-495a-bbde-c541e6c490e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:16.650588 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69677bc-9320-495a-bbde-c541e6c490e0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:16.667727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 309/652] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:16.668666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69677bc-9320-495a-bbde-c541e6c490e0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:16.685311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51390fd-d607-465c-bee2-7247a10d40dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:16.686532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51390fd-d607-465c-bee2-7247a10d40dd None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.687074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51390fd-d607-465c-bee2-7247a10d40dd None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.687074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51390fd-d607-465c-bee2-7247a10d40dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.693734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51390fd-d607-465c-bee2-7247a10d40dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:16.695163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51390fd-d607-465c-bee2-7247a10d40dd None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:16.697690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-f51390fd-d607-465c-bee2-7247a10d40dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:16.710623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:16.711591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:16.712580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:16.712580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:16.712580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:16.712580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:16.712580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:16.712580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:16.712580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:16.712580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:16.712580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 344/653] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:16.712580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51390fd-d607-465c-bee2-7247a10d40dd None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:16.772344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69677bc-9320-495a-bbde-c541e6c490e0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:16.774684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:16.806530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104650b-0579-4fd3-8fd0-9d23bcf3de7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:16.807279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104650b-0579-4fd3-8fd0-9d23bcf3de7e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.807656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104650b-0579-4fd3-8fd0-9d23bcf3de7e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.807906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104650b-0579-4fd3-8fd0-9d23bcf3de7e None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.808617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104650b-0579-4fd3-8fd0-9d23bcf3de7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:16.808952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104650b-0579-4fd3-8fd0-9d23bcf3de7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:16.810305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104650b-0579-4fd3-8fd0-9d23bcf3de7e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:16.812630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51390fd-d607-465c-bee2-7247a10d40dd None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:16.822290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 310/654] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:16.823094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104650b-0579-4fd3-8fd0-9d23bcf3de7e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:16.847500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dd91e-42fc-42c1-b29a-10af261b9a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:16.848186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dd91e-42fc-42c1-b29a-10af261b9a82 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.848471 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dd91e-42fc-42c1-b29a-10af261b9a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.849925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dd91e-42fc-42c1-b29a-10af261b9a82 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.850787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dd91e-42fc-42c1-b29a-10af261b9a82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:16.851103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dd91e-42fc-42c1-b29a-10af261b9a82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:16.853101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dd91e-42fc-42c1-b29a-10af261b9a82 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:16.873414 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 345/655] 10.222.0.67 () {56 vars in 1137 bytes} [Thu Jun 23 11:42:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:16.875438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659adbde-2597-4422-ae9a-07f7c3b82c7d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.875782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659adbde-2597-4422-ae9a-07f7c3b82c7d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.876114 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659adbde-2597-4422-ae9a-07f7c3b82c7d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.877683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 346/656] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:16.878286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659adbde-2597-4422-ae9a-07f7c3b82c7d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:16.912580 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:16.928420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104650b-0579-4fd3-8fd0-9d23bcf3de7e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:16.961859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27479fad-8686-4f50-a15e-7f780fa66243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:16.962571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27479fad-8686-4f50-a15e-7f780fa66243 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:16.962864 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27479fad-8686-4f50-a15e-7f780fa66243 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:16.963143 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27479fad-8686-4f50-a15e-7f780fa66243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:16.973117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27479fad-8686-4f50-a15e-7f780fa66243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:16.974566 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27479fad-8686-4f50-a15e-7f780fa66243 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:16.979275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659adbde-2597-4422-ae9a-07f7c3b82c7d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-27479fad-8686-4f50-a15e-7f780fa66243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:16.993181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:16.994210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:16.995307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:16.995307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:16.995307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:16.995307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:16.995307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 311/657] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:16.995307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27479fad-8686-4f50-a15e-7f780fa66243 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:17.005259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:17.017787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b5c5f-2c5d-4994-bafd-6f13880e3dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:17.018454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b5c5f-2c5d-4994-bafd-6f13880e3dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.018744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b5c5f-2c5d-4994-bafd-6f13880e3dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.019025 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b5c5f-2c5d-4994-bafd-6f13880e3dd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.019813 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b5c5f-2c5d-4994-bafd-6f13880e3dd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:17.020127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b5c5f-2c5d-4994-bafd-6f13880e3dd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:17.021608 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7b5c5f-2c5d-4994-bafd-6f13880e3dd2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:17.033403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 347/658] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:17.034199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b5c5f-2c5d-4994-bafd-6f13880e3dd2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:17.096696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27479fad-8686-4f50-a15e-7f780fa66243 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:17.130541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f528b6-926b-40b2-b52e-3e11d4c9dfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:17.131331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f528b6-926b-40b2-b52e-3e11d4c9dfb5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.131622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f528b6-926b-40b2-b52e-3e11d4c9dfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.131898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f528b6-926b-40b2-b52e-3e11d4c9dfb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.137322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b5c5f-2c5d-4994-bafd-6f13880e3dd2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:17.138869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f528b6-926b-40b2-b52e-3e11d4c9dfb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:17.141075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f528b6-926b-40b2-b52e-3e11d4c9dfb5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:17.142932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-30f528b6-926b-40b2-b52e-3e11d4c9dfb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:17.156224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:17.157661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:17.158562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:17.158562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:17.158562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:17.158562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:17.158562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:17.158562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:17.158562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:17.158562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:17.158562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 312/659] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:17.159047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41930a-20b1-4294-8bd5-c12d545983ae None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.159239 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41930a-20b1-4294-8bd5-c12d545983ae None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.159448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41930a-20b1-4294-8bd5-c12d545983ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.175318 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3a41930a-20b1-4294-8bd5-c12d545983ae None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:17.176282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5f4e8-aa75-40c8-ba1a-d8a9e708b47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:17.176521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3a41930a-20b1-4294-8bd5-c12d545983ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:17.176969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5f4e8-aa75-40c8-ba1a-d8a9e708b47c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.177255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5f4e8-aa75-40c8-ba1a-d8a9e708b47c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.177535 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5f4e8-aa75-40c8-ba1a-d8a9e708b47c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.229793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:17.246515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 313/660] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:17.247362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41930a-20b1-4294-8bd5-c12d545983ae None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:17.255102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5f4e8-aa75-40c8-ba1a-d8a9e708b47c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:17.257006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5f4e8-aa75-40c8-ba1a-d8a9e708b47c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:17.345311 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41930a-20b1-4294-8bd5-c12d545983ae None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:17.362467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 348/661] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:42:17.362467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5f4e8-aa75-40c8-ba1a-d8a9e708b47c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:17.377983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:17.386667 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02426957-5d44-4360-b175-657c84df9c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:17.387389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02426957-5d44-4360-b175-657c84df9c12 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.387680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02426957-5d44-4360-b175-657c84df9c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.387987 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02426957-5d44-4360-b175-657c84df9c12 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.388744 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02426957-5d44-4360-b175-657c84df9c12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:17.389062 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02426957-5d44-4360-b175-657c84df9c12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:17.391001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02426957-5d44-4360-b175-657c84df9c12 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:17.402310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 314/662] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:17.403119 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02426957-5d44-4360-b175-657c84df9c12 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:17.440109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:17.459939 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5f4e8-aa75-40c8-ba1a-d8a9e708b47c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:17.493479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2d3c4-5d70-48bd-814f-b864c57ba750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:17.494058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2d3c4-5d70-48bd-814f-b864c57ba750 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.494340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2d3c4-5d70-48bd-814f-b864c57ba750 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.494616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2d3c4-5d70-48bd-814f-b864c57ba750 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.495434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2d3c4-5d70-48bd-814f-b864c57ba750 None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=abcbb404d9a148568a82697abd51ec30) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:17.495745 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2d3c4-5d70-48bd-814f-b864c57ba750 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:17.497197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2d3c4-5d70-48bd-814f-b864c57ba750 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:17.503663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 349/663] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:17 2022] GET /identity/v3/role_assignments?user.id=abcbb404d9a148568a82697abd51ec30&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:17.504355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2d3c4-5d70-48bd-814f-b864c57ba750 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:17.519339 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02426957-5d44-4360-b175-657c84df9c12 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:17.560743 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdbd1b-8b99-4edf-86e7-f28a10644be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:17.561421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdbd1b-8b99-4edf-86e7-f28a10644be0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.561707 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdbd1b-8b99-4edf-86e7-f28a10644be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.561993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdbd1b-8b99-4edf-86e7-f28a10644be0 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.562813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdbd1b-8b99-4edf-86e7-f28a10644be0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:17.563149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdbd1b-8b99-4edf-86e7-f28a10644be0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:17.564617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befdbd1b-8b99-4edf-86e7-f28a10644be0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:17.579353 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 315/664] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:17.582363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdbd1b-8b99-4edf-86e7-f28a10644be0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:17.622108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2d3c4-5d70-48bd-814f-b864c57ba750 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:17.638247 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:17.669132 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cadb8-acdb-4308-b33a-c00b8c8c2bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:17.670152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cadb8-acdb-4308-b33a-c00b8c8c2bac None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.670246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cadb8-acdb-4308-b33a-c00b8c8c2bac None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.670525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5cadb8-acdb-4308-b33a-c00b8c8c2bac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.685854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5cadb8-acdb-4308-b33a-c00b8c8c2bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4f5cadb8-acdb-4308-b33a-c00b8c8c2bac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:17.693925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:17.695376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:17.696275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:17.696275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:17.696275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:17.696275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:17.696275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:17.696275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:17.696275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:17.696275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:17.696275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:17.696275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:17.696275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:17.696275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:17.697536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 350/665] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:17.698965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cadb8-acdb-4308-b33a-c00b8c8c2bac None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:17.709584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:17.713309 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befdbd1b-8b99-4edf-86e7-f28a10644be0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:17.750821 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7cf70-7df7-4711-b274-79e972d9981b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:17.751558 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7cf70-7df7-4711-b274-79e972d9981b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.751822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7cf70-7df7-4711-b274-79e972d9981b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.752097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7cf70-7df7-4711-b274-79e972d9981b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.764767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7cf70-7df7-4711-b274-79e972d9981b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:17.766151 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c7cf70-7df7-4711-b274-79e972d9981b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-95c7cf70-7df7-4711-b274-79e972d9981b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:17.787832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:17.788947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:17.790033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:17.790033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:17.790033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:17.790033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:17.790033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 316/666] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:17.793614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7cf70-7df7-4711-b274-79e972d9981b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:17.812430 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5cadb8-acdb-4308-b33a-c00b8c8c2bac None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:17.848726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd805c-23a0-42ed-b54a-7ce1530bdb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:17.854760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd805c-23a0-42ed-b54a-7ce1530bdb96 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.854760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd805c-23a0-42ed-b54a-7ce1530bdb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.854760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd805c-23a0-42ed-b54a-7ce1530bdb96 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.863986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd805c-23a0-42ed-b54a-7ce1530bdb96 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:17.865779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd805c-23a0-42ed-b54a-7ce1530bdb96 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-77bd805c-23a0-42ed-b54a-7ce1530bdb96 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:17.881627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:17.882867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:17.888934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:17.888934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:17.888934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:17.888934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:17.888934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:17.888934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:17.888934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:17.888934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:17.888934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 11:42:17.888934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:17.888934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 351/667] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:17.888934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd805c-23a0-42ed-b54a-7ce1530bdb96 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:17.911210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:17.914681 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7cf70-7df7-4711-b274-79e972d9981b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:17.955731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b291ce8-0072-49fc-bd45-5a03538b43f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:17.956744 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b291ce8-0072-49fc-bd45-5a03538b43f0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:17.957231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b291ce8-0072-49fc-bd45-5a03538b43f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:17.957513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b291ce8-0072-49fc-bd45-5a03538b43f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:17.958359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b291ce8-0072-49fc-bd45-5a03538b43f0 None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=36c18bbd22f74efbae7aab1cdd3f6403) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:17.958679 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b291ce8-0072-49fc-bd45-5a03538b43f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:17.960266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b291ce8-0072-49fc-bd45-5a03538b43f0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:17.960983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:17.976707 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 317/668] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:17 2022] GET /identity/v3/role_assignments?user.id=36c18bbd22f74efbae7aab1cdd3f6403&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:17.976707 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b291ce8-0072-49fc-bd45-5a03538b43f0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:18.012941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd805c-23a0-42ed-b54a-7ce1530bdb96 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:18.060400 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8438c3-a079-44dd-9c83-a1355d04cd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:18.061179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8438c3-a079-44dd-9c83-a1355d04cd4c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:18.061466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8438c3-a079-44dd-9c83-a1355d04cd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:18.061746 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8438c3-a079-44dd-9c83-a1355d04cd4c None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:18.062509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8438c3-a079-44dd-9c83-a1355d04cd4c None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:18.062822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8438c3-a079-44dd-9c83-a1355d04cd4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:18.064239 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8438c3-a079-44dd-9c83-a1355d04cd4c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:18.074259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 352/669] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:18.075150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8438c3-a079-44dd-9c83-a1355d04cd4c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:18.098413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b291ce8-0072-49fc-bd45-5a03538b43f0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:18.136834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673b833-a740-4c1e-b7ae-f84bec027746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:18.137498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673b833-a740-4c1e-b7ae-f84bec027746 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:18.137781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673b833-a740-4c1e-b7ae-f84bec027746 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:18.138057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673b833-a740-4c1e-b7ae-f84bec027746 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:18.138780 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673b833-a740-4c1e-b7ae-f84bec027746 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:18.139318 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673b833-a740-4c1e-b7ae-f84bec027746 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:18.141599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3673b833-a740-4c1e-b7ae-f84bec027746 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:18.158804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 318/670] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:18.159594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673b833-a740-4c1e-b7ae-f84bec027746 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:18.188322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:18.201496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8438c3-a079-44dd-9c83-a1355d04cd4c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:18.238498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:18.249442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ae3bb-0efb-4c69-9859-b9e9624dcf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:18.250217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ae3bb-0efb-4c69-9859-b9e9624dcf08 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:18.250578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ae3bb-0efb-4c69-9859-b9e9624dcf08 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:18.250952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ae3bb-0efb-4c69-9859-b9e9624dcf08 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:18.251816 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ae3bb-0efb-4c69-9859-b9e9624dcf08 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:18.252134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ae3bb-0efb-4c69-9859-b9e9624dcf08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:18.253618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ae3bb-0efb-4c69-9859-b9e9624dcf08 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:18.275769 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 353/671] 10.222.0.67 () {56 vars in 1139 bytes} [Thu Jun 23 11:42:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:18.280858 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ae3bb-0efb-4c69-9859-b9e9624dcf08 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:18.302775 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3673b833-a740-4c1e-b7ae-f84bec027746 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:18.350158 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21225f-8825-4acc-869b-331ab7e6d507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:18.351606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21225f-8825-4acc-869b-331ab7e6d507 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:18.352658 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21225f-8825-4acc-869b-331ab7e6d507 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:18.353142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21225f-8825-4acc-869b-331ab7e6d507 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:18.367920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21225f-8825-4acc-869b-331ab7e6d507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:18.369514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21225f-8825-4acc-869b-331ab7e6d507 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:18.393496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ae3bb-0efb-4c69-9859-b9e9624dcf08 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-1d21225f-8825-4acc-869b-331ab7e6d507 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:18.393813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:18.394663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:18.395553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:18.395553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:18.395553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:18.395553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:18.395553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 319/672] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:18.400819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21225f-8825-4acc-869b-331ab7e6d507 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:18.438296 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0b97c-14ff-44e1-9ba9-bebf58ffcbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:18.439189 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0b97c-14ff-44e1-9ba9-bebf58ffcbfe None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:18.439548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0b97c-14ff-44e1-9ba9-bebf58ffcbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:18.439976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0b97c-14ff-44e1-9ba9-bebf58ffcbfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:18.447842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0b97c-14ff-44e1-9ba9-bebf58ffcbfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:18.450003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0b97c-14ff-44e1-9ba9-bebf58ffcbfe None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-99b0b97c-14ff-44e1-9ba9-bebf58ffcbfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:18.465054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:18.466003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:18.466895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:18.466895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:18.466895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:18.466895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:18.466895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:18.466895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:18.466895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:18.466895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:18.466895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 354/673] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:18.468024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0b97c-14ff-44e1-9ba9-bebf58ffcbfe None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:18.510244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:18.520363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:18.547180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21225f-8825-4acc-869b-331ab7e6d507 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:18.584446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0b97c-14ff-44e1-9ba9-bebf58ffcbfe None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:18.588798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b91088-c919-48ca-a44d-59c1e98319f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:18.589461 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b91088-c919-48ca-a44d-59c1e98319f2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:18.589752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b91088-c919-48ca-a44d-59c1e98319f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:18.590039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b91088-c919-48ca-a44d-59c1e98319f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:18.630295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db286bf-7134-4c13-9cc3-858fe98af63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:18.631265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db286bf-7134-4c13-9cc3-858fe98af63f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:18.632255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db286bf-7134-4c13-9cc3-858fe98af63f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:18.632739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db286bf-7134-4c13-9cc3-858fe98af63f None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:18.633701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db286bf-7134-4c13-9cc3-858fe98af63f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:18.634655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db286bf-7134-4c13-9cc3-858fe98af63f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:18.636613 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db286bf-7134-4c13-9cc3-858fe98af63f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:18.649596 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 355/674] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:18.650679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db286bf-7134-4c13-9cc3-858fe98af63f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:18.692535 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b91088-c919-48ca-a44d-59c1e98319f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:18.695190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b91088-c919-48ca-a44d-59c1e98319f2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:18.779423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db286bf-7134-4c13-9cc3-858fe98af63f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:18.787643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:18.788605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:18.816692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 320/675] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:42:18.818342 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b91088-c919-48ca-a44d-59c1e98319f2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:18.821884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554ecde-f10a-448d-bb95-15473f368581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:18.823043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554ecde-f10a-448d-bb95-15473f368581 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:18.823043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554ecde-f10a-448d-bb95-15473f368581 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:18.823254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554ecde-f10a-448d-bb95-15473f368581 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:18.823986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554ecde-f10a-448d-bb95-15473f368581 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:18.824310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554ecde-f10a-448d-bb95-15473f368581 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:18.825783 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554ecde-f10a-448d-bb95-15473f368581 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:18.839855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 356/676] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:18.840806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554ecde-f10a-448d-bb95-15473f368581 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:18.922992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b91088-c919-48ca-a44d-59c1e98319f2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:18.951445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554ecde-f10a-448d-bb95-15473f368581 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:18.959091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667dd7d-137b-4748-a6c6-0d7d709ad02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:18.960132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667dd7d-137b-4748-a6c6-0d7d709ad02d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:18.960649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667dd7d-137b-4748-a6c6-0d7d709ad02d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:18.961150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667dd7d-137b-4748-a6c6-0d7d709ad02d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:18.968666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667dd7d-137b-4748-a6c6-0d7d709ad02d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:18.970375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667dd7d-137b-4748-a6c6-0d7d709ad02d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-3667dd7d-137b-4748-a6c6-0d7d709ad02d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:18.985965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:18.986923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:18.987872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:18.987872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:18.987872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:18.987872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:18.987872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:18.987872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:18.987872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:18.987872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:18.987872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:42:18.987872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:18.990184 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a1ce-7b2b-48d5-8528-5e9674c0e718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:18.990680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 321/677] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:42:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:18.991647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667dd7d-137b-4748-a6c6-0d7d709ad02d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:18.995937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a1ce-7b2b-48d5-8528-5e9674c0e718 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:18.995937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a1ce-7b2b-48d5-8528-5e9674c0e718 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:18.996163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab28a1ce-7b2b-48d5-8528-5e9674c0e718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:19.008712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28a1ce-7b2b-48d5-8528-5e9674c0e718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:19.010400 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab28a1ce-7b2b-48d5-8528-5e9674c0e718 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:19.028341 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-ab28a1ce-7b2b-48d5-8528-5e9674c0e718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:19.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:19.031506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:19.032741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:19.032741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:19.032741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:19.032741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:19.035045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 357/678] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:19.036116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a1ce-7b2b-48d5-8528-5e9674c0e718 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:19.059690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:19.099228 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667dd7d-137b-4748-a6c6-0d7d709ad02d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:19.134467 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fb264-69b5-4e52-814b-67c88afcc4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:19.135123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fb264-69b5-4e52-814b-67c88afcc4f7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:19.135406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fb264-69b5-4e52-814b-67c88afcc4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:19.135682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1fb264-69b5-4e52-814b-67c88afcc4f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:19.136508 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fb264-69b5-4e52-814b-67c88afcc4f7 None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=74fdc12d728d45018ea668a4073da3b8) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:19.136859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fb264-69b5-4e52-814b-67c88afcc4f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:19.138376 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1fb264-69b5-4e52-814b-67c88afcc4f7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:19.145198 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 322/679] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:18 2022] GET /identity/v3/role_assignments?user.id=74fdc12d728d45018ea668a4073da3b8&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:19.146161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fb264-69b5-4e52-814b-67c88afcc4f7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:19.149923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab28a1ce-7b2b-48d5-8528-5e9674c0e718 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:19.218133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5fd35-a8b8-4f79-83c8-c2a0f44657eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:19.219273 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5fd35-a8b8-4f79-83c8-c2a0f44657eb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:19.219769 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5fd35-a8b8-4f79-83c8-c2a0f44657eb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:19.220376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5fd35-a8b8-4f79-83c8-c2a0f44657eb None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:19.221475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5fd35-a8b8-4f79-83c8-c2a0f44657eb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:19.222002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5fd35-a8b8-4f79-83c8-c2a0f44657eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:19.223772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5fd35-a8b8-4f79-83c8-c2a0f44657eb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:19.249721 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 358/680] 10.222.0.67 () {56 vars in 1135 bytes} [Thu Jun 23 11:42:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:19.252023 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5fd35-a8b8-4f79-83c8-c2a0f44657eb None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:19.254317 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:19.257740 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1fb264-69b5-4e52-814b-67c88afcc4f7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:19.295303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8ca47-e66d-4db0-8d8a-09cb99630c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:19.296458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8ca47-e66d-4db0-8d8a-09cb99630c91 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:19.304703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8ca47-e66d-4db0-8d8a-09cb99630c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:19.304703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8ca47-e66d-4db0-8d8a-09cb99630c91 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:19.304703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8ca47-e66d-4db0-8d8a-09cb99630c91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:19.304703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8ca47-e66d-4db0-8d8a-09cb99630c91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:19.304703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8ca47-e66d-4db0-8d8a-09cb99630c91 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:19.318444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 323/681] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:19.319894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8ca47-e66d-4db0-8d8a-09cb99630c91 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:19.391016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:19.428691 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5fd35-a8b8-4f79-83c8-c2a0f44657eb None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:19.470453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8ca47-e66d-4db0-8d8a-09cb99630c91 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:19.482253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd98f3-bed4-42b7-b514-4618f49848c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:19.482982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd98f3-bed4-42b7-b514-4618f49848c8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:19.483776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd98f3-bed4-42b7-b514-4618f49848c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:19.484059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd98f3-bed4-42b7-b514-4618f49848c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:19.514260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c8112-862c-4646-a98c-a9f480a19881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:19.515058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c8112-862c-4646-a98c-a9f480a19881 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:19.515340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c8112-862c-4646-a98c-a9f480a19881 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:19.515615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c8112-862c-4646-a98c-a9f480a19881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:19.522934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c8112-862c-4646-a98c-a9f480a19881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:19.524361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c8112-862c-4646-a98c-a9f480a19881 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f59c8112-862c-4646-a98c-a9f480a19881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:19.540370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:19.541561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:19.542485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:19.542485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:19.542485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:19.542485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:19.542485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:19.542485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:19.542485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:19.542485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:19.542485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 324/682] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:19.548031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c8112-862c-4646-a98c-a9f480a19881 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:19.574957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:19.581540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd98f3-bed4-42b7-b514-4618f49848c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:19.583765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd98f3-bed4-42b7-b514-4618f49848c8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:19.663208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c8112-862c-4646-a98c-a9f480a19881 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:19.688489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:19.704304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7ea51-3572-4077-8612-793721e08936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:19.705079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7ea51-3572-4077-8612-793721e08936 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:19.705462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7ea51-3572-4077-8612-793721e08936 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:19.705835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7ea51-3572-4077-8612-793721e08936 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:19.706724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7ea51-3572-4077-8612-793721e08936 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:19.708082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7ea51-3572-4077-8612-793721e08936 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:19.709761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7ea51-3572-4077-8612-793721e08936 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:19.725393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 359/683] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 11:42:19.727293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 325/684] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:19.735401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd98f3-bed4-42b7-b514-4618f49848c8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:19.737264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7ea51-3572-4077-8612-793721e08936 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:19.849000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7ea51-3572-4077-8612-793721e08936 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:19.857218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd98f3-bed4-42b7-b514-4618f49848c8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:19.873124 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:19.888354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ad5f2-c4a4-49d4-9d10-b8a895357baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:19.889768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ad5f2-c4a4-49d4-9d10-b8a895357baa None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:19.890454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ad5f2-c4a4-49d4-9d10-b8a895357baa None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:19.890846 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ad5f2-c4a4-49d4-9d10-b8a895357baa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:19.892073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ad5f2-c4a4-49d4-9d10-b8a895357baa None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=5f095d3370be457cb6687ac8f8c9f8ab) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:19.892496 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ad5f2-c4a4-49d4-9d10-b8a895357baa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:19.894884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ad5f2-c4a4-49d4-9d10-b8a895357baa None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:19.902138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a3288-4e20-4e1c-b6a9-ccda67395391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:19.903360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 326/685] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:19 2022] GET /identity/v3/role_assignments?user.id=5f095d3370be457cb6687ac8f8c9f8ab&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:19.903985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a3288-4e20-4e1c-b6a9-ccda67395391 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:19.904447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a3288-4e20-4e1c-b6a9-ccda67395391 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:19.905457 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a3288-4e20-4e1c-b6a9-ccda67395391 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:19.906077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59f7e-3978-4c3e-a9ff-9d3293acc8ed None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:19.906797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59f7e-3978-4c3e-a9ff-9d3293acc8ed None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:19.907182 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f59f7e-3978-4c3e-a9ff-9d3293acc8ed None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:19.916804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 327/686] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:19.916804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dec05-545a-4e1f-8dbe-b7211980b77a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:19.916804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dec05-545a-4e1f-8dbe-b7211980b77a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:19.916804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3dec05-545a-4e1f-8dbe-b7211980b77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:19.945427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a3288-4e20-4e1c-b6a9-ccda67395391 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:19.947149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a3288-4e20-4e1c-b6a9-ccda67395391 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:19.948956 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9f3dec05-545a-4e1f-8dbe-b7211980b77a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:19.953709 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9f3dec05-545a-4e1f-8dbe-b7211980b77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-714a3288-4e20-4e1c-b6a9-ccda67395391 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:19.980883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:19.982051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:19.983077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:19.983077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:19.983077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:19.983077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:19.996555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 360/687] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:20.002272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a3288-4e20-4e1c-b6a9-ccda67395391 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:20.074586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:20.085973 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 328/688] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:42:20.108702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dec05-545a-4e1f-8dbe-b7211980b77a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:20.158806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a3288-4e20-4e1c-b6a9-ccda67395391 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:20.197554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:20.203590 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462890e7-f36d-4710-92f2-6c83de1a33bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:20.204976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462890e7-f36d-4710-92f2-6c83de1a33bd None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:20.205434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462890e7-f36d-4710-92f2-6c83de1a33bd None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:20.205862 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462890e7-f36d-4710-92f2-6c83de1a33bd None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:20.207166 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462890e7-f36d-4710-92f2-6c83de1a33bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:20.207772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462890e7-f36d-4710-92f2-6c83de1a33bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:20.209682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462890e7-f36d-4710-92f2-6c83de1a33bd None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:20.226550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 361/689] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:20.233177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462890e7-f36d-4710-92f2-6c83de1a33bd None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:20.234781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3dec05-545a-4e1f-8dbe-b7211980b77a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:20.280990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53193e0b-3b2d-42f2-88bf-76385517e520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:20.281814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53193e0b-3b2d-42f2-88bf-76385517e520 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:20.282103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53193e0b-3b2d-42f2-88bf-76385517e520 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:20.282651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53193e0b-3b2d-42f2-88bf-76385517e520 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:20.283317 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53193e0b-3b2d-42f2-88bf-76385517e520 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:20.284709 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53193e0b-3b2d-42f2-88bf-76385517e520 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-53193e0b-3b2d-42f2-88bf-76385517e520 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:20.313099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:20.314148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:42:20.315072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:20.318654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 329/690] 10.222.0.67 () {56 vars in 1129 bytes} [Thu Jun 23 11:42:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:20.327248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53193e0b-3b2d-42f2-88bf-76385517e520 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:20.380955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:20.396736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462890e7-f36d-4710-92f2-6c83de1a33bd None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:20.435208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527689f6-f56c-47ba-8bb7-78af2c2b1268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:20.436891 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527689f6-f56c-47ba-8bb7-78af2c2b1268 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:20.437372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527689f6-f56c-47ba-8bb7-78af2c2b1268 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:20.437719 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527689f6-f56c-47ba-8bb7-78af2c2b1268 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:20.444949 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527689f6-f56c-47ba-8bb7-78af2c2b1268 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:20.446472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527689f6-f56c-47ba-8bb7-78af2c2b1268 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-527689f6-f56c-47ba-8bb7-78af2c2b1268 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:20.462617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:20.463802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:20.464757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:20.464757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:20.464757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:20.464757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:20.464757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:20.464757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:20.464757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:20.464757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:20.464757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 362/691] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:20.464757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da66a1-7e36-4cb3-a178-86efc7655c5c None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:20.464757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da66a1-7e36-4cb3-a178-86efc7655c5c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:20.465467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da66a1-7e36-4cb3-a178-86efc7655c5c None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:20.466055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 363/692] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:20.472427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da66a1-7e36-4cb3-a178-86efc7655c5c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:20.492915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53193e0b-3b2d-42f2-88bf-76385517e520 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:20.527898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf26435-9ebb-44e2-a4b0-013f303513d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:20.528564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf26435-9ebb-44e2-a4b0-013f303513d9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:20.528882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf26435-9ebb-44e2-a4b0-013f303513d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:20.529161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf26435-9ebb-44e2-a4b0-013f303513d9 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:20.530008 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf26435-9ebb-44e2-a4b0-013f303513d9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:20.530530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf26435-9ebb-44e2-a4b0-013f303513d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:20.533308 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf26435-9ebb-44e2-a4b0-013f303513d9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:20.546192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 330/693] 10.222.0.67 () {56 vars in 1124 bytes} [Thu Jun 23 11:42:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:20.550036 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09544af1-866f-4836-804b-4fd39462af9c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:20.550638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09544af1-866f-4836-804b-4fd39462af9c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:20.550638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09544af1-866f-4836-804b-4fd39462af9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:20.556762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:20.571025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-09544af1-866f-4836-804b-4fd39462af9c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:20.572251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-09544af1-866f-4836-804b-4fd39462af9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:20.597879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da66a1-7e36-4cb3-a178-86efc7655c5c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:20.638770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fbe3a-4003-4c7a-922a-f460c4b1d88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:20.639468 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fbe3a-4003-4c7a-922a-f460c4b1d88a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:20.639761 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fbe3a-4003-4c7a-922a-f460c4b1d88a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:20.640042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2fbe3a-4003-4c7a-922a-f460c4b1d88a None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:20.640872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fbe3a-4003-4c7a-922a-f460c4b1d88a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:20.641188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fbe3a-4003-4c7a-922a-f460c4b1d88a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:20.642725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2fbe3a-4003-4c7a-922a-f460c4b1d88a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:20.643335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 331/694] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:20.644196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09544af1-866f-4836-804b-4fd39462af9c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:20.675122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 364/695] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:20.678830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fbe3a-4003-4c7a-922a-f460c4b1d88a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:20.697048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:20.783589 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09544af1-866f-4836-804b-4fd39462af9c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:20.799189 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2fbe3a-4003-4c7a-922a-f460c4b1d88a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:20.830279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:20.837495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf69046-2f8f-4fa7-a160-8d22c46ae6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:20.837701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc41b55-1eba-4a27-91ac-4975447aa585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:20.838514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf69046-2f8f-4fa7-a160-8d22c46ae6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:20.838633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc41b55-1eba-4a27-91ac-4975447aa585 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:20.838826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc41b55-1eba-4a27-91ac-4975447aa585 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:20.839113 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc41b55-1eba-4a27-91ac-4975447aa585 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:20.839322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf69046-2f8f-4fa7-a160-8d22c46ae6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:20.839747 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf69046-2f8f-4fa7-a160-8d22c46ae6b0 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:20.840610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf69046-2f8f-4fa7-a160-8d22c46ae6b0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:20.841330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf69046-2f8f-4fa7-a160-8d22c46ae6b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:20.843794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf69046-2f8f-4fa7-a160-8d22c46ae6b0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:20.851140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc41b55-1eba-4a27-91ac-4975447aa585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:20.851614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 332/696] 10.222.0.67 () {56 vars in 1124 bytes} [Thu Jun 23 11:42:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:20.852654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf69046-2f8f-4fa7-a160-8d22c46ae6b0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8dc41b55-1eba-4a27-91ac-4975447aa585 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:20.859793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:20.861360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:20.862431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:20.862431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:20.862431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:20.862431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:20.862431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:20.862431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:20.862431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:20.862431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:20.862431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:20.862431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:20.862431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:42:20.862431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:20.863717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 365/697] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:42:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:20.875019 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc41b55-1eba-4a27-91ac-4975447aa585 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:20.989986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf69046-2f8f-4fa7-a160-8d22c46ae6b0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:21.028516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:21.053374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea693d6-9093-44ff-b9cc-2a18e2e0b10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:21.053964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea693d6-9093-44ff-b9cc-2a18e2e0b10a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:21.054249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea693d6-9093-44ff-b9cc-2a18e2e0b10a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:21.054530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea693d6-9093-44ff-b9cc-2a18e2e0b10a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:21.055381 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea693d6-9093-44ff-b9cc-2a18e2e0b10a None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=eb9b9af5e0b44e34837e1cdb5ff6d33e) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:21.055698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea693d6-9093-44ff-b9cc-2a18e2e0b10a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:21.058476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea693d6-9093-44ff-b9cc-2a18e2e0b10a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:21.071418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 333/698] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:20 2022] GET /identity/v3/role_assignments?user.id=eb9b9af5e0b44e34837e1cdb5ff6d33e&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:21.091556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc41b55-1eba-4a27-91ac-4975447aa585 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:21.131313 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658edd2-4f58-4794-894b-b4b8a78014be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:21.132326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658edd2-4f58-4794-894b-b4b8a78014be None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:21.132827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658edd2-4f58-4794-894b-b4b8a78014be None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:21.133311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658edd2-4f58-4794-894b-b4b8a78014be None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:21.134267 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658edd2-4f58-4794-894b-b4b8a78014be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:21.134764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658edd2-4f58-4794-894b-b4b8a78014be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:21.136351 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658edd2-4f58-4794-894b-b4b8a78014be None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:21.146730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 366/699] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:21.152630 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea693d6-9093-44ff-b9cc-2a18e2e0b10a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:21.223027 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:21.275242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea693d6-9093-44ff-b9cc-2a18e2e0b10a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:21.314946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4b70e-4687-448a-871e-b95b116cb9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:21.315673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4b70e-4687-448a-871e-b95b116cb9fa None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:21.315958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4b70e-4687-448a-871e-b95b116cb9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:21.316238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4b70e-4687-448a-871e-b95b116cb9fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:21.327766 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4b70e-4687-448a-871e-b95b116cb9fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:21.329276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4b70e-4687-448a-871e-b95b116cb9fa None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-adb4b70e-4687-448a-871e-b95b116cb9fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:21.348659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:21.349636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:21.350541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:21.350541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:21.350541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:21.350541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:21.350541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 334/700] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:21.356779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658edd2-4f58-4794-894b-b4b8a78014be None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:21.470139 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658edd2-4f58-4794-894b-b4b8a78014be None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:21.500930 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:21.511301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b776b-c70e-4c12-9e82-5ee99d6582f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:21.511976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b776b-c70e-4c12-9e82-5ee99d6582f2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:21.512264 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b776b-c70e-4c12-9e82-5ee99d6582f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:21.512543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b776b-c70e-4c12-9e82-5ee99d6582f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:21.513444 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b776b-c70e-4c12-9e82-5ee99d6582f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:21.513759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b776b-c70e-4c12-9e82-5ee99d6582f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:21.515170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b776b-c70e-4c12-9e82-5ee99d6582f2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:21.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 367/701] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:21.534542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4b70e-4687-448a-871e-b95b116cb9fa None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:21.655233 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4b70e-4687-448a-871e-b95b116cb9fa None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:21.689027 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:21.711824 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a25a6-2f77-46cd-9b98-d536164845ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:21.712626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a25a6-2f77-46cd-9b98-d536164845ba None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:21.712946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a25a6-2f77-46cd-9b98-d536164845ba None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:21.713227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a25a6-2f77-46cd-9b98-d536164845ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:21.725132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a25a6-2f77-46cd-9b98-d536164845ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:21.726580 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a25a6-2f77-46cd-9b98-d536164845ba None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f87a25a6-2f77-46cd-9b98-d536164845ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:21.747439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:21.749023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:21.749971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:21.749971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:21.749971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:21.749971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:21.749971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:21.749971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:21.749971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:21.749971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:21.749971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:42:21.749971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:21.756532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 335/702] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 23 11:42:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:21.762389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b776b-c70e-4c12-9e82-5ee99d6582f2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:21.899952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019a280-23b9-4a1a-9432-9aff9337a66a None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:21.900281 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019a280-23b9-4a1a-9432-9aff9337a66a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:21.900521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019a280-23b9-4a1a-9432-9aff9337a66a None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:21.903107 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 336/703] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:21.907185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb3fe7-c87b-4a9d-ad01-7de87b356ed8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:21.907955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb3fe7-c87b-4a9d-ad01-7de87b356ed8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:21.908433 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb3fe7-c87b-4a9d-ad01-7de87b356ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:21.926860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b776b-c70e-4c12-9e82-5ee99d6582f2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:21.938714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-60bb3fe7-c87b-4a9d-ad01-7de87b356ed8 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:21.940270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-60bb3fe7-c87b-4a9d-ad01-7de87b356ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:21.965409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a59e68-9159-4b0f-929d-1924d86d6019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:21.966090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a59e68-9159-4b0f-929d-1924d86d6019 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:21.966381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a59e68-9159-4b0f-929d-1924d86d6019 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:21.966665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a59e68-9159-4b0f-929d-1924d86d6019 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:21.967711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a59e68-9159-4b0f-929d-1924d86d6019 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:21.968028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a59e68-9159-4b0f-929d-1924d86d6019 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:21.969656 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a59e68-9159-4b0f-929d-1924d86d6019 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:21.990861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 368/704] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 23 11:42:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:21.996085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a59e68-9159-4b0f-929d-1924d86d6019 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:22.054768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:22.115231 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a59e68-9159-4b0f-929d-1924d86d6019 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:22.122906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 337/705] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:22.151963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdcf6d-6914-4cdc-984b-022fa8a1c996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:22.152617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdcf6d-6914-4cdc-984b-022fa8a1c996 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:22.152941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdcf6d-6914-4cdc-984b-022fa8a1c996 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:22.153284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdcf6d-6914-4cdc-984b-022fa8a1c996 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:22.161940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb3fe7-c87b-4a9d-ad01-7de87b356ed8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:22.231774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdcf6d-6914-4cdc-984b-022fa8a1c996 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:22.233989 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdcf6d-6914-4cdc-984b-022fa8a1c996 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:22.297375 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:22.353097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 369/706] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 11:42:22.361690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdcf6d-6914-4cdc-984b-022fa8a1c996 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:22.391158 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb3fe7-c87b-4a9d-ad01-7de87b356ed8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:22.393260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:22.430417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33eb85-5ac2-42a2-b2f6-297500800697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:22.431548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33eb85-5ac2-42a2-b2f6-297500800697 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:22.432026 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33eb85-5ac2-42a2-b2f6-297500800697 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:22.432513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33eb85-5ac2-42a2-b2f6-297500800697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:22.444508 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33eb85-5ac2-42a2-b2f6-297500800697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:22.446377 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33eb85-5ac2-42a2-b2f6-297500800697 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-3c33eb85-5ac2-42a2-b2f6-297500800697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:22.466213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:22.467144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:22.468076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:22.468076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:22.468076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:22.468076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:22.470476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 338/707] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:22.472079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac1ebe-827b-463f-bd7d-295f5846eb9b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:22.472565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac1ebe-827b-463f-bd7d-295f5846eb9b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:22.473087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac1ebe-827b-463f-bd7d-295f5846eb9b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:22.474970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 339/708] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:22.476006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac1ebe-827b-463f-bd7d-295f5846eb9b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:22.494269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdcf6d-6914-4cdc-984b-022fa8a1c996 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:22.530925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c071b5e-9e65-4e92-b355-cae21d6ce851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:22.531726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c071b5e-9e65-4e92-b355-cae21d6ce851 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:22.532070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c071b5e-9e65-4e92-b355-cae21d6ce851 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:22.532616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c071b5e-9e65-4e92-b355-cae21d6ce851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:22.548867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c071b5e-9e65-4e92-b355-cae21d6ce851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:22.550462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c071b5e-9e65-4e92-b355-cae21d6ce851 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:22.564772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-5c071b5e-9e65-4e92-b355-cae21d6ce851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:22.579426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:22.580418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:22.581371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:22.581371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:22.581371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:22.581371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:22.581371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 370/709] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:22.582685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b9a86-0c05-4cf7-bfec-15908b40b7e1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:22.582973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b9a86-0c05-4cf7-bfec-15908b40b7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:22.583259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b9a86-0c05-4cf7-bfec-15908b40b7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:22.604585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4a5b9a86-0c05-4cf7-bfec-15908b40b7e1 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:22.606136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4a5b9a86-0c05-4cf7-bfec-15908b40b7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:22.607235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac1ebe-827b-463f-bd7d-295f5846eb9b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:22.653916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c05da6-eb2c-4534-b79d-76fd3c7219a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:22.654692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c05da6-eb2c-4534-b79d-76fd3c7219a3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:22.655102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c05da6-eb2c-4534-b79d-76fd3c7219a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:22.655396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c05da6-eb2c-4534-b79d-76fd3c7219a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:22.656168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c05da6-eb2c-4534-b79d-76fd3c7219a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:22.656917 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c05da6-eb2c-4534-b79d-76fd3c7219a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:22.658612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c05da6-eb2c-4534-b79d-76fd3c7219a3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:22.675640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 340/710] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:22.677455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c05da6-eb2c-4534-b79d-76fd3c7219a3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:22.689204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 371/711] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:22.690419 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b9a86-0c05-4cf7-bfec-15908b40b7e1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:22.699351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:22.807518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b9a86-0c05-4cf7-bfec-15908b40b7e1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:22.813065 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c05da6-eb2c-4534-b79d-76fd3c7219a3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:22.839844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:22.850759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b296-1bc0-4c6b-b071-1d8ea1eab6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:22.851874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b296-1bc0-4c6b-b071-1d8ea1eab6ba None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:22.851985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d6378-dda2-4214-8f18-6da3e29adc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:22.852524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b296-1bc0-4c6b-b071-1d8ea1eab6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:22.852660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d6378-dda2-4214-8f18-6da3e29adc8b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:22.852886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d6378-dda2-4214-8f18-6da3e29adc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:22.853170 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d6378-dda2-4214-8f18-6da3e29adc8b None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:22.853550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1b296-1bc0-4c6b-b071-1d8ea1eab6ba None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:22.853931 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d6378-dda2-4214-8f18-6da3e29adc8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:22.854245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d6378-dda2-4214-8f18-6da3e29adc8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:22.854921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1b296-1bc0-4c6b-b071-1d8ea1eab6ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:22.855678 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d6378-dda2-4214-8f18-6da3e29adc8b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:22.856840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee1b296-1bc0-4c6b-b071-1d8ea1eab6ba None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:22.870474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 341/712] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:22.871445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d6378-dda2-4214-8f18-6da3e29adc8b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:22.886140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 372/713] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:42:22 2022] POST /identity/v3/users => generated 254 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:22.887493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b296-1bc0-4c6b-b071-1d8ea1eab6ba None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:22.994881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:23.023525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee1b296-1bc0-4c6b-b071-1d8ea1eab6ba None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:23.038550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d6378-dda2-4214-8f18-6da3e29adc8b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:23.062430 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f5981-152d-49d3-b487-d72d1696c99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:23.063347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5981-152d-49d3-b487-d72d1696c99b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:23.063630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5981-152d-49d3-b487-d72d1696c99b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:23.063906 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5981-152d-49d3-b487-d72d1696c99b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:23.071246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f5981-152d-49d3-b487-d72d1696c99b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:23.072795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f5981-152d-49d3-b487-d72d1696c99b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-441f5981-152d-49d3-b487-d72d1696c99b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:23.089300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:23.090307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:23.091336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:23.091336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:23.091336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:23.091336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:23.091336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:23.091336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:23.091336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:23.091336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:23.091336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 373/714] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:23.097419 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f5981-152d-49d3-b487-d72d1696c99b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:23.099024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edeaca-1b3a-4627-940e-92f3740ade0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:23.100039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edeaca-1b3a-4627-940e-92f3740ade0f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:23.100877 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edeaca-1b3a-4627-940e-92f3740ade0f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:23.101383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edeaca-1b3a-4627-940e-92f3740ade0f None admin] PATH_INFO: `/v3/services/network` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:23.102346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edeaca-1b3a-4627-940e-92f3740ade0f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:23.103968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edeaca-1b3a-4627-940e-92f3740ade0f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-41edeaca-1b3a-4627-940e-92f3740ade0f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:23.123296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:23.124390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:42:23.125276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:23.138665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 342/715] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:22 2022] GET /identity/v3/services/network => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:23.144715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edeaca-1b3a-4627-940e-92f3740ade0f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:23.185834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:23.258402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f5981-152d-49d3-b487-d72d1696c99b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:23.291510 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edeaca-1b3a-4627-940e-92f3740ade0f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:23.309699 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d888d54-6f4b-4bc1-ac3b-4e144647b3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:23.311927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d888d54-6f4b-4bc1-ac3b-4e144647b3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:23.312429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d888d54-6f4b-4bc1-ac3b-4e144647b3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:23.312957 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d888d54-6f4b-4bc1-ac3b-4e144647b3d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:23.313959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d888d54-6f4b-4bc1-ac3b-4e144647b3d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:23.314497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d888d54-6f4b-4bc1-ac3b-4e144647b3d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:23.317007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d888d54-6f4b-4bc1-ac3b-4e144647b3d8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:23.332782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 374/716] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:23.339036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d888d54-6f4b-4bc1-ac3b-4e144647b3d8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:23.339369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c45d6-938c-4a9f-8145-963bf1d89c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:23.341873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c45d6-938c-4a9f-8145-963bf1d89c11 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:23.342803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c45d6-938c-4a9f-8145-963bf1d89c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:23.343298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c45d6-938c-4a9f-8145-963bf1d89c11 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:23.345304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c45d6-938c-4a9f-8145-963bf1d89c11 None admin] RBAC: Adding query filter params (name=network) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:23.345850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c45d6-938c-4a9f-8145-963bf1d89c11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:23.347488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c45d6-938c-4a9f-8145-963bf1d89c11 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:23.354025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 343/717] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:23.362080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c45d6-938c-4a9f-8145-963bf1d89c11 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:23.366343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:23.480783 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d888d54-6f4b-4bc1-ac3b-4e144647b3d8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:23.490827 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c45d6-938c-4a9f-8145-963bf1d89c11 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:23.491021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:23.531572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52377972-5895-498e-be93-28da3dbbe335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:23.532155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52377972-5895-498e-be93-28da3dbbe335 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:23.533144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52377972-5895-498e-be93-28da3dbbe335 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:23.533425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52377972-5895-498e-be93-28da3dbbe335 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:23.534276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52377972-5895-498e-be93-28da3dbbe335 None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=36c18bbd22f74efbae7aab1cdd3f6403) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:23.534598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52377972-5895-498e-be93-28da3dbbe335 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:23.536098 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52377972-5895-498e-be93-28da3dbbe335 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:23.539663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3d4b8-2760-40f2-8050-29f5429e0872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:23.540896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3d4b8-2760-40f2-8050-29f5429e0872 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:23.541422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3d4b8-2760-40f2-8050-29f5429e0872 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:23.542397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3d4b8-2760-40f2-8050-29f5429e0872 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:23.543316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3d4b8-2760-40f2-8050-29f5429e0872 None admin] RBAC: Adding query filter params (type=network) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:23.543800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3d4b8-2760-40f2-8050-29f5429e0872 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:23.548726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3d4b8-2760-40f2-8050-29f5429e0872 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:23.550531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 375/718] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:23 2022] GET /identity/v3/role_assignments?user.id=36c18bbd22f74efbae7aab1cdd3f6403&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=569707378794437b93d168fc8bb9cbc3 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:23.551947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf55954-75a7-4d7a-a478-aba71d507847 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:23.552303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf55954-75a7-4d7a-a478-aba71d507847 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:23.552656 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf55954-75a7-4d7a-a478-aba71d507847 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:23.554392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 376/719] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:23.559764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 344/720] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:23.562570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac1dbc-e56b-4b0c-88e3-70bb18800e64 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:23.562938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac1dbc-e56b-4b0c-88e3-70bb18800e64 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:23.563294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac1dbc-e56b-4b0c-88e3-70bb18800e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:23.601954 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-33ac1dbc-e56b-4b0c-88e3-70bb18800e64 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:23.603192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-33ac1dbc-e56b-4b0c-88e3-70bb18800e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:23.735710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 377/721] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:23.761842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3d4b8-2760-40f2-8050-29f5429e0872 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:23.840340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:23.895652 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3d4b8-2760-40f2-8050-29f5429e0872 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:23.896183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3e644-3c85-4a6d-a3e1-c282cb0e8ffc None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:23.896485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3e644-3c85-4a6d-a3e1-c282cb0e8ffc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:23.896813 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3e644-3c85-4a6d-a3e1-c282cb0e8ffc None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:23.901030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 378/722] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:23.904002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698d0d2-744c-4274-bd0b-1eb455b56a28 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:23.904129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698d0d2-744c-4274-bd0b-1eb455b56a28 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:23.904415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698d0d2-744c-4274-bd0b-1eb455b56a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:23.931573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c698d0d2-744c-4274-bd0b-1eb455b56a28 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:23.934223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c698d0d2-744c-4274-bd0b-1eb455b56a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:23.966848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e11ef-0db7-4be9-a296-630dae64d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:23.968110 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e11ef-0db7-4be9-a296-630dae64d2e2 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:23.968394 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e11ef-0db7-4be9-a296-630dae64d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:23.968691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e11ef-0db7-4be9-a296-630dae64d2e2 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:23.969439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e11ef-0db7-4be9-a296-630dae64d2e2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:23.970812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e11ef-0db7-4be9-a296-630dae64d2e2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:23.996492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 345/723] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:42:23 2022] POST /identity/v3/services => generated 251 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:24.057380 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 379/724] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:23 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:42:24.078927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e11ef-0db7-4be9-a296-630dae64d2e2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:24.213940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e11ef-0db7-4be9-a296-630dae64d2e2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:24.244665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:24.261836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151cd39-1e72-457c-90a9-a327dd496279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:24.262657 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151cd39-1e72-457c-90a9-a327dd496279 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:24.262947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151cd39-1e72-457c-90a9-a327dd496279 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:24.263234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151cd39-1e72-457c-90a9-a327dd496279 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:24.278147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151cd39-1e72-457c-90a9-a327dd496279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-4151cd39-1e72-457c-90a9-a327dd496279 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:24.286754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:24.287690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:24.288531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 346/725] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:42:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:24.294022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698d0d2-744c-4274-bd0b-1eb455b56a28 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:24.328016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:24.411037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698d0d2-744c-4274-bd0b-1eb455b56a28 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:24.448348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdb711-f08f-4457-adfe-771ca565f370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:24.449426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdb711-f08f-4457-adfe-771ca565f370 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:24.449934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdb711-f08f-4457-adfe-771ca565f370 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:24.450449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdb711-f08f-4457-adfe-771ca565f370 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:24.451417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdb711-f08f-4457-adfe-771ca565f370 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:24.452454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdb711-f08f-4457-adfe-771ca565f370 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:24.454062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdb711-f08f-4457-adfe-771ca565f370 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:24.464076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 380/726] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:24.471562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151cd39-1e72-457c-90a9-a327dd496279 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:24.606410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151cd39-1e72-457c-90a9-a327dd496279 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:24.647272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a3ace-7254-487b-b709-a995eadb9ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:24.648068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a3ace-7254-487b-b709-a995eadb9ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:24.648362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a3ace-7254-487b-b709-a995eadb9ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:24.648643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21a3ace-7254-487b-b709-a995eadb9ea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:24.663839 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a3ace-7254-487b-b709-a995eadb9ea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:24.663839 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21a3ace-7254-487b-b709-a995eadb9ea5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b21a3ace-7254-487b-b709-a995eadb9ea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:24.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:24.695982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:24.696914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:24.696914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:24.696914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:24.696914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:24.696914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 347/727] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:24.702194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdb711-f08f-4457-adfe-771ca565f370 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:24.950476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:24.963670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdb711-f08f-4457-adfe-771ca565f370 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:25.009620 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e1822-0d2e-4638-98d8-9e2bb5f481f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:25.010281 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e1822-0d2e-4638-98d8-9e2bb5f481f3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:25.010560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e1822-0d2e-4638-98d8-9e2bb5f481f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:25.010833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e1822-0d2e-4638-98d8-9e2bb5f481f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:25.011551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e1822-0d2e-4638-98d8-9e2bb5f481f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:25.011863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e1822-0d2e-4638-98d8-9e2bb5f481f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:25.013249 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e1822-0d2e-4638-98d8-9e2bb5f481f3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:25.027853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 381/728] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:25.034420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a3ace-7254-487b-b709-a995eadb9ea5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:25.089595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:25.162318 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21a3ace-7254-487b-b709-a995eadb9ea5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:25.201103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd4ccf-7bfb-41a8-8f1c-e55153b48cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:25.202361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd4ccf-7bfb-41a8-8f1c-e55153b48cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:25.202650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd4ccf-7bfb-41a8-8f1c-e55153b48cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:25.202932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd4ccf-7bfb-41a8-8f1c-e55153b48cd9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:25.218705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd4ccf-7bfb-41a8-8f1c-e55153b48cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:25.220217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fd4ccf-7bfb-41a8-8f1c-e55153b48cd9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-63fd4ccf-7bfb-41a8-8f1c-e55153b48cd9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:25.238102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:25.239100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:25.240473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:25.240473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:25.240473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:25.240473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:25.240473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:25.240473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:25.240473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:25.240473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:25.240473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:42:25.240473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:25.240473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 348/729] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:42:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:25.248140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e1822-0d2e-4638-98d8-9e2bb5f481f3 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:25.395995 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e1822-0d2e-4638-98d8-9e2bb5f481f3 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:25.430944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abea93-a34e-438f-af2c-4bafd7ce7bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:25.432020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abea93-a34e-438f-af2c-4bafd7ce7bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:25.432525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abea93-a34e-438f-af2c-4bafd7ce7bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:25.433031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abea93-a34e-438f-af2c-4bafd7ce7bb7 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:25.434062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abea93-a34e-438f-af2c-4bafd7ce7bb7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:25.434590 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abea93-a34e-438f-af2c-4bafd7ce7bb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:25.436870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abea93-a34e-438f-af2c-4bafd7ce7bb7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:25.455962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 382/730] 10.222.0.67 () {56 vars in 1135 bytes} [Thu Jun 23 11:42:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:25.461039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd4ccf-7bfb-41a8-8f1c-e55153b48cd9 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:25.601972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:25.605643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fd4ccf-7bfb-41a8-8f1c-e55153b48cd9 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:25.653420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2c4ed-3649-49e9-9893-ee862ad46664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:25.654090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2c4ed-3649-49e9-9893-ee862ad46664 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:25.654384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2c4ed-3649-49e9-9893-ee862ad46664 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:25.654736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2c4ed-3649-49e9-9893-ee862ad46664 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:25.768213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2c4ed-3649-49e9-9893-ee862ad46664 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:25.769703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd2c4ed-3649-49e9-9893-ee862ad46664 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:25.903066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:25.911804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 349/731] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:25 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 11:42:25.916937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abea93-a34e-438f-af2c-4bafd7ce7bb7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:25.927641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:25.928712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cd3b5-b262-46f3-8fee-60fa0923cb1a None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:25.929002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cd3b5-b262-46f3-8fee-60fa0923cb1a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:25.929284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cd3b5-b262-46f3-8fee-60fa0923cb1a None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:25.933067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 350/732] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:25.936521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d035f4-fb75-42e0-87eb-cbc2186b7072 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:25.937598 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d035f4-fb75-42e0-87eb-cbc2186b7072 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:25.937886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d035f4-fb75-42e0-87eb-cbc2186b7072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:25.967779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-77d035f4-fb75-42e0-87eb-cbc2186b7072 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:25.969040 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-77d035f4-fb75-42e0-87eb-cbc2186b7072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:26.090385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abea93-a34e-438f-af2c-4bafd7ce7bb7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:26.101012 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 351/733] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 11:42:26.130151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f14d1-4781-4ce5-a040-a8a8cb78ce25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:26.130993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f14d1-4781-4ce5-a040-a8a8cb78ce25 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.131281 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f14d1-4781-4ce5-a040-a8a8cb78ce25 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.131554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f14d1-4781-4ce5-a040-a8a8cb78ce25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.137817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d035f4-fb75-42e0-87eb-cbc2186b7072 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:26.153697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f14d1-4781-4ce5-a040-a8a8cb78ce25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:26.156324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f14d1-4781-4ce5-a040-a8a8cb78ce25 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-753f14d1-4781-4ce5-a040-a8a8cb78ce25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:26.175912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:26.176953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:26.177886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:26.177886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:26.177886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:26.177886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:26.177886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 383/734] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:26.179583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b533042-71f9-4e5e-912a-769bf4a1c50d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.179583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b533042-71f9-4e5e-912a-769bf4a1c50d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.179583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b533042-71f9-4e5e-912a-769bf4a1c50d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.185081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 384/735] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:26.186055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b533042-71f9-4e5e-912a-769bf4a1c50d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:26.258151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:26.263242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d035f4-fb75-42e0-87eb-cbc2186b7072 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:26.293140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:26.303741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490dac0e-6956-41c2-9d77-e69e4f6c330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:26.304608 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490dac0e-6956-41c2-9d77-e69e4f6c330c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.305053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490dac0e-6956-41c2-9d77-e69e4f6c330c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.305460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490dac0e-6956-41c2-9d77-e69e4f6c330c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.308280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b533042-71f9-4e5e-912a-769bf4a1c50d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:26.317372 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490dac0e-6956-41c2-9d77-e69e4f6c330c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-490dac0e-6956-41c2-9d77-e69e4f6c330c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:26.326046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:26.327219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:26.328089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:26.328089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:26.328089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:26.328089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:26.328089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:26.328089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:26.328089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:26.328089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:26.328089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:26.328089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:26.328089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:26.328089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:26.328758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 352/736] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:26.330379 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4fc11-0f53-46e9-9aed-9a791532d338 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.330735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4fc11-0f53-46e9-9aed-9a791532d338 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.331085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4fc11-0f53-46e9-9aed-9a791532d338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.348997 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1b7d2-04b6-4c7e-a431-224daf03e175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:26.349960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1b7d2-04b6-4c7e-a431-224daf03e175 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.350433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1b7d2-04b6-4c7e-a431-224daf03e175 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.350542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f1a4fc11-0f53-46e9-9aed-9a791532d338 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:26.351010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1b7d2-04b6-4c7e-a431-224daf03e175 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.351946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f1a4fc11-0f53-46e9-9aed-9a791532d338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:26.352060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1b7d2-04b6-4c7e-a431-224daf03e175 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:26.352577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1b7d2-04b6-4c7e-a431-224daf03e175 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:26.354162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1b7d2-04b6-4c7e-a431-224daf03e175 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:26.368268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 385/737] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:26.369501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1b7d2-04b6-4c7e-a431-224daf03e175 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:26.434531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 353/738] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 11:42:26.435578 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4fc11-0f53-46e9-9aed-9a791532d338 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:26.482203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1b7d2-04b6-4c7e-a431-224daf03e175 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:26.501835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:26.522420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e2011-c94e-425c-818f-3bc298ab91e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:26.523096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e2011-c94e-425c-818f-3bc298ab91e1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.523380 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e2011-c94e-425c-818f-3bc298ab91e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.523658 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e2011-c94e-425c-818f-3bc298ab91e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.524405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e2011-c94e-425c-818f-3bc298ab91e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:26.524738 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e2011-c94e-425c-818f-3bc298ab91e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:26.526105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e2011-c94e-425c-818f-3bc298ab91e1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:26.538904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 386/739] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:26.540393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb32b9-ca4f-46d3-b0d4-f004c93b9046 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.540712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb32b9-ca4f-46d3-b0d4-f004c93b9046 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.541001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb32b9-ca4f-46d3-b0d4-f004c93b9046 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.542387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 387/740] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:26.547093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb32b9-ca4f-46d3-b0d4-f004c93b9046 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:26.564015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4fc11-0f53-46e9-9aed-9a791532d338 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:26.582735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:26.602009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a573434-16c2-4c1d-9933-7ed8368ae236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:26.603284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a573434-16c2-4c1d-9933-7ed8368ae236 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.603724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a573434-16c2-4c1d-9933-7ed8368ae236 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.604196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a573434-16c2-4c1d-9933-7ed8368ae236 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.612184 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a573434-16c2-4c1d-9933-7ed8368ae236 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:26.614002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a573434-16c2-4c1d-9933-7ed8368ae236 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-1a573434-16c2-4c1d-9933-7ed8368ae236 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:26.628793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:26.629793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:26.630832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:26.630832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:26.630832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:26.630832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:26.630832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:26.630832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:26.630832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:26.630832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:26.630832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 354/741] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:26.631323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a573434-16c2-4c1d-9933-7ed8368ae236 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:26.655769 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb32b9-ca4f-46d3-b0d4-f004c93b9046 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:26.692983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380a3ca-59a7-48be-bce3-127087d977a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:26.693654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380a3ca-59a7-48be-bce3-127087d977a6 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.693939 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380a3ca-59a7-48be-bce3-127087d977a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.694222 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380a3ca-59a7-48be-bce3-127087d977a6 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.694961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380a3ca-59a7-48be-bce3-127087d977a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:26.696333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380a3ca-59a7-48be-bce3-127087d977a6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:26.712430 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 388/742] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:42:26 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:26.714356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89096f82-cb0f-4b9c-8565-d763a860c62b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.714356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89096f82-cb0f-4b9c-8565-d763a860c62b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.714540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89096f82-cb0f-4b9c-8565-d763a860c62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.745333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-89096f82-cb0f-4b9c-8565-d763a860c62b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:26.745616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a573434-16c2-4c1d-9933-7ed8368ae236 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:26.747085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-89096f82-cb0f-4b9c-8565-d763a860c62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:26.792946 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:26.796742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368a1b0-bc39-4237-b722-50b9604cdfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:26.797618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368a1b0-bc39-4237-b722-50b9604cdfc4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.797912 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368a1b0-bc39-4237-b722-50b9604cdfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.798192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368a1b0-bc39-4237-b722-50b9604cdfc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.812961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368a1b0-bc39-4237-b722-50b9604cdfc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:26.814513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368a1b0-bc39-4237-b722-50b9604cdfc4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:26.831577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 389/743] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:26.833397 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf89f00-8b12-4a0e-8233-4a78e75b433f None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.833912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf89f00-8b12-4a0e-8233-4a78e75b433f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0368a1b0-bc39-4237-b722-50b9604cdfc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:26.834037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:26.834851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:26.835758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:26.835758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:26.835758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:26.835758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:26.835758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 355/744] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:26.836481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368a1b0-bc39-4237-b722-50b9604cdfc4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:26.837050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf89f00-8b12-4a0e-8233-4a78e75b433f None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.838977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 390/745] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:26.844281 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ab7a3-62ec-434a-b5de-3110c3eaa0d0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.845013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ab7a3-62ec-434a-b5de-3110c3eaa0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.845563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ab7a3-62ec-434a-b5de-3110c3eaa0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.866974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-083ab7a3-62ec-434a-b5de-3110c3eaa0d0 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:26.868111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-083ab7a3-62ec-434a-b5de-3110c3eaa0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:26.877509 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:26.947926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 391/746] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:26.950371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ab7a3-62ec-434a-b5de-3110c3eaa0d0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:26.960700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368a1b0-bc39-4237-b722-50b9604cdfc4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:26.996943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aec121-15e6-4093-ab15-313d10d5b380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:26.997606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aec121-15e6-4093-ab15-313d10d5b380 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:26.997893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aec121-15e6-4093-ab15-313d10d5b380 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:26.998175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aec121-15e6-4093-ab15-313d10d5b380 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:26.999617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aec121-15e6-4093-ab15-313d10d5b380 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:26.999951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aec121-15e6-4093-ab15-313d10d5b380 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:27.001439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aec121-15e6-4093-ab15-313d10d5b380 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:27.015038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 356/747] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:27.015875 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aec121-15e6-4093-ab15-313d10d5b380 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:27.066117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:27.069642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ab7a3-62ec-434a-b5de-3110c3eaa0d0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:27.106371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca6da0-f1c7-4a0e-bf84-330ed4302de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:27.107239 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca6da0-f1c7-4a0e-bf84-330ed4302de0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:27.107664 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:27.108578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca6da0-f1c7-4a0e-bf84-330ed4302de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:27.108896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca6da0-f1c7-4a0e-bf84-330ed4302de0 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:27.109591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca6da0-f1c7-4a0e-bf84-330ed4302de0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:27.109933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca6da0-f1c7-4a0e-bf84-330ed4302de0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:27.111559 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca6da0-f1c7-4a0e-bf84-330ed4302de0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:27.124446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 392/748] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:27.125222 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca6da0-f1c7-4a0e-bf84-330ed4302de0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:27.126916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aec121-15e6-4093-ab15-313d10d5b380 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:27.165177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af71ab-d4ff-4506-a796-c62a0545b249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:27.165978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af71ab-d4ff-4506-a796-c62a0545b249 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:27.166265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af71ab-d4ff-4506-a796-c62a0545b249 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:27.166548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af71ab-d4ff-4506-a796-c62a0545b249 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:27.177816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af71ab-d4ff-4506-a796-c62a0545b249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f4af71ab-d4ff-4506-a796-c62a0545b249 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:27.185590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:27.187190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:27.188029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 357/749] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:42:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:27.188845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af71ab-d4ff-4506-a796-c62a0545b249 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:27.242966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca6da0-f1c7-4a0e-bf84-330ed4302de0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:27.279994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e99e-91fd-4474-9644-18830251a28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:27.280844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e99e-91fd-4474-9644-18830251a28e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:27.281138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e99e-91fd-4474-9644-18830251a28e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:27.281454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e99e-91fd-4474-9644-18830251a28e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:27.282123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8e99e-91fd-4474-9644-18830251a28e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:27.283764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8e99e-91fd-4474-9644-18830251a28e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-59f8e99e-91fd-4474-9644-18830251a28e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:27.298678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:27.299660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:27.301164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 393/750] 10.222.0.67 () {56 vars in 1129 bytes} [Thu Jun 23 11:42:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:27.302039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e99e-91fd-4474-9644-18830251a28e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:27.305353 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af71ab-d4ff-4506-a796-c62a0545b249 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:27.328283 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:27.344779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab985f-fa65-441b-8896-aacefa429866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:27.344998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab985f-fa65-441b-8896-aacefa429866 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:27.345272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab985f-fa65-441b-8896-aacefa429866 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:27.345646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab985f-fa65-441b-8896-aacefa429866 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:27.346516 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab985f-fa65-441b-8896-aacefa429866 None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=eb9b9af5e0b44e34837e1cdb5ff6d33e) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:27.346907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab985f-fa65-441b-8896-aacefa429866 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:27.348434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab985f-fa65-441b-8896-aacefa429866 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:27.355446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 358/751] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:27 2022] GET /identity/v3/role_assignments?user.id=eb9b9af5e0b44e34837e1cdb5ff6d33e&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=569707378794437b93d168fc8bb9cbc3 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:27.355446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab985f-fa65-441b-8896-aacefa429866 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:27.390459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:27.454024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e99e-91fd-4474-9644-18830251a28e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:27.504071 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2be11-1829-4d93-92bc-c8db7af5edda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:27.505333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2be11-1829-4d93-92bc-c8db7af5edda None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:27.505728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2be11-1829-4d93-92bc-c8db7af5edda None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:27.506111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2be11-1829-4d93-92bc-c8db7af5edda None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:27.506498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab985f-fa65-441b-8896-aacefa429866 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:27.517638 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2be11-1829-4d93-92bc-c8db7af5edda None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:27.520024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e2be11-1829-4d93-92bc-c8db7af5edda None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-37e2be11-1829-4d93-92bc-c8db7af5edda None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:27.541992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:27.543008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:27.543976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:27.543976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:27.543976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:27.543976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:27.543976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:27.543976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:27.543976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:27.543976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:27.543976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:42:27.543976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:27.543976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 394/752] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 23 11:42:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:27.544772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2be11-1829-4d93-92bc-c8db7af5edda None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:27.548636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10df18-b48c-4a59-a095-90e628401773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:27.549463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10df18-b48c-4a59-a095-90e628401773 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:27.549746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10df18-b48c-4a59-a095-90e628401773 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:27.550020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10df18-b48c-4a59-a095-90e628401773 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:27.550890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10df18-b48c-4a59-a095-90e628401773 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:27.551202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10df18-b48c-4a59-a095-90e628401773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:27.553642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10df18-b48c-4a59-a095-90e628401773 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:27.564751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 359/753] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:27.565691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10df18-b48c-4a59-a095-90e628401773 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:27.641368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:27.644031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:27.675119 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2be11-1829-4d93-92bc-c8db7af5edda None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:27.680984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10df18-b48c-4a59-a095-90e628401773 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:27.711837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad96926-045b-4b7f-892b-ffb2377134d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:27.712516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad96926-045b-4b7f-892b-ffb2377134d5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:27.712839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad96926-045b-4b7f-892b-ffb2377134d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:27.713127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad96926-045b-4b7f-892b-ffb2377134d5 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:27.713902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad96926-045b-4b7f-892b-ffb2377134d5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:27.714153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad96926-045b-4b7f-892b-ffb2377134d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:27.715542 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad96926-045b-4b7f-892b-ffb2377134d5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:27.716411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdc522-fcac-4db5-ab07-b0d038c010c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:27.717782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdc522-fcac-4db5-ab07-b0d038c010c0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:27.718159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdc522-fcac-4db5-ab07-b0d038c010c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:27.718440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdc522-fcac-4db5-ab07-b0d038c010c0 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:27.719280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdc522-fcac-4db5-ab07-b0d038c010c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:27.719595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdc522-fcac-4db5-ab07-b0d038c010c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:27.721046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdc522-fcac-4db5-ab07-b0d038c010c0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:27.726067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 395/754] 10.222.0.67 () {56 vars in 1124 bytes} [Thu Jun 23 11:42:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:27.726984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad96926-045b-4b7f-892b-ffb2377134d5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:27.740590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 360/755] 10.222.0.67 () {56 vars in 1149 bytes} [Thu Jun 23 11:42:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:27.741465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdc522-fcac-4db5-ab07-b0d038c010c0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:27.843718 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad96926-045b-4b7f-892b-ffb2377134d5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:27.850696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdc522-fcac-4db5-ab07-b0d038c010c0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:27.883299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785c92d-09c3-408f-8658-aae1e29f769e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:27.884203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785c92d-09c3-408f-8658-aae1e29f769e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:27.884494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785c92d-09c3-408f-8658-aae1e29f769e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:27.884909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785c92d-09c3-408f-8658-aae1e29f769e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:27.889988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:27.897175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24c754-97e7-400f-991d-d7f58c52e828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:27.898278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24c754-97e7-400f-991d-d7f58c52e828 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:27.898574 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24c754-97e7-400f-991d-d7f58c52e828 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:27.898853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24c754-97e7-400f-991d-d7f58c52e828 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:27.899861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24c754-97e7-400f-991d-d7f58c52e828 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:27.900152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785c92d-09c3-408f-8658-aae1e29f769e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:27.900325 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24c754-97e7-400f-991d-d7f58c52e828 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:27.901561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24c754-97e7-400f-991d-d7f58c52e828 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:27.901744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b785c92d-09c3-408f-8658-aae1e29f769e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:27.911694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 361/756] 10.222.0.67 () {56 vars in 1124 bytes} [Thu Jun 23 11:42:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:27.912558 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24c754-97e7-400f-991d-d7f58c52e828 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:27.917278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-b785c92d-09c3-408f-8658-aae1e29f769e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:27.934209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:27.936509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:27.938752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:27.938752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:27.938752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:27.938752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:27.938752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 396/757] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:27.938752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785c92d-09c3-408f-8658-aae1e29f769e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:28.037313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24c754-97e7-400f-991d-d7f58c52e828 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:28.061621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785c92d-09c3-408f-8658-aae1e29f769e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:28.074455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43ec50-4030-44d7-bd4b-61d06a0c9c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:28.075273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43ec50-4030-44d7-bd4b-61d06a0c9c6d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.075560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43ec50-4030-44d7-bd4b-61d06a0c9c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.075841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe43ec50-4030-44d7-bd4b-61d06a0c9c6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.101382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf451d-83e0-416e-bc55-2ff491ff829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:28.102044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf451d-83e0-416e-bc55-2ff491ff829f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.102334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf451d-83e0-416e-bc55-2ff491ff829f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.102622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcf451d-83e0-416e-bc55-2ff491ff829f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.103422 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf451d-83e0-416e-bc55-2ff491ff829f None admin] RBAC: Adding query filter params (interface=public, service_id=1a8ed6c9b8de4c338eaf0e2feaed3b3e, region_id=RegionOne) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:28.103741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf451d-83e0-416e-bc55-2ff491ff829f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:28.105146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcf451d-83e0-416e-bc55-2ff491ff829f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:28.117042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 397/758] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 23 11:42:27 2022] GET /identity/v3/endpoints?service_id=1a8ed6c9b8de4c338eaf0e2feaed3b3e&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:28.117903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf451d-83e0-416e-bc55-2ff491ff829f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:28.148762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:28.160613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43ec50-4030-44d7-bd4b-61d06a0c9c6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:28.162079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe43ec50-4030-44d7-bd4b-61d06a0c9c6d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:28.189512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:28.231140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcf451d-83e0-416e-bc55-2ff491ff829f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:28.268492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d7c3e-1447-4f00-84d3-7721775d3342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:28.269363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d7c3e-1447-4f00-84d3-7721775d3342 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.269752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d7c3e-1447-4f00-84d3-7721775d3342 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.270134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d7c3e-1447-4f00-84d3-7721775d3342 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.270963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d7c3e-1447-4f00-84d3-7721775d3342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:28.271377 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d7c3e-1447-4f00-84d3-7721775d3342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:28.272889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d7c3e-1447-4f00-84d3-7721775d3342 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:28.290111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 398/759] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:28.291164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d7c3e-1447-4f00-84d3-7721775d3342 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:28.292333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 362/760] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:42:28.293210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43ec50-4030-44d7-bd4b-61d06a0c9c6d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:28.396368 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:28.407411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe43ec50-4030-44d7-bd4b-61d06a0c9c6d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:28.423511 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d7c3e-1447-4f00-84d3-7721775d3342 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:28.450339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:28.461556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f92ff3-2d94-47b1-b1b6-b34468a238b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:28.462353 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f92ff3-2d94-47b1-b1b6-b34468a238b0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.462639 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f92ff3-2d94-47b1-b1b6-b34468a238b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.462917 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f92ff3-2d94-47b1-b1b6-b34468a238b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.463348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b52e65-2eb3-4d64-aee4-0c30aebbceab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:28.464092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b52e65-2eb3-4d64-aee4-0c30aebbceab None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.464476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b52e65-2eb3-4d64-aee4-0c30aebbceab None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.464874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b52e65-2eb3-4d64-aee4-0c30aebbceab None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.465643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b52e65-2eb3-4d64-aee4-0c30aebbceab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:28.467644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b52e65-2eb3-4d64-aee4-0c30aebbceab None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:28.472450 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f92ff3-2d94-47b1-b1b6-b34468a238b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:28.473998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f92ff3-2d94-47b1-b1b6-b34468a238b0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:28.481161 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 399/761] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 23 11:42:28 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:28.482344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b52e65-2eb3-4d64-aee4-0c30aebbceab None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-73f92ff3-2d94-47b1-b1b6-b34468a238b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:28.492656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:28.495064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:28.497386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:28.497386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:28.497386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:28.497386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:28.497386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:28.497386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:28.497386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:28.497386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:28.497386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:42:28.497386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:28.497386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 363/762] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 23 11:42:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:28.503777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f92ff3-2d94-47b1-b1b6-b34468a238b0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:28.689484 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b52e65-2eb3-4d64-aee4-0c30aebbceab None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:28.695630 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f92ff3-2d94-47b1-b1b6-b34468a238b0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:28.726188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c3a92-b677-47f3-9692-1e7dbae2f128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:28.726980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c3a92-b677-47f3-9692-1e7dbae2f128 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.727309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c3a92-b677-47f3-9692-1e7dbae2f128 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.727599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c3a92-b677-47f3-9692-1e7dbae2f128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.733156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0dd9f6-44a2-410f-8f11-2a217c0aaf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:28.733881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0dd9f6-44a2-410f-8f11-2a217c0aaf70 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.734169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0dd9f6-44a2-410f-8f11-2a217c0aaf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.734447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0dd9f6-44a2-410f-8f11-2a217c0aaf70 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.735397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0dd9f6-44a2-410f-8f11-2a217c0aaf70 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:28.735708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0dd9f6-44a2-410f-8f11-2a217c0aaf70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:28.737287 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0dd9f6-44a2-410f-8f11-2a217c0aaf70 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:28.738662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c3a92-b677-47f3-9692-1e7dbae2f128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:28.740112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c3a92-b677-47f3-9692-1e7dbae2f128 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:28.755996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 364/763] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 23 11:42:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-600c3a92-b677-47f3-9692-1e7dbae2f128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:28.760445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:28.762407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:28.763662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:28.763662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:28.763662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:28.763662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:28.763947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0dd9f6-44a2-410f-8f11-2a217c0aaf70 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:28.764632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 400/764] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:28.774218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c3a92-b677-47f3-9692-1e7dbae2f128 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:28.782803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:28.824423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:28.877433 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0dd9f6-44a2-410f-8f11-2a217c0aaf70 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:28.903411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c3a92-b677-47f3-9692-1e7dbae2f128 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:28.913878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13806d4-4f06-4df5-8280-549978c6d0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:28.914669 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13806d4-4f06-4df5-8280-549978c6d0ed None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.914953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13806d4-4f06-4df5-8280-549978c6d0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.915233 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13806d4-4f06-4df5-8280-549978c6d0ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.926043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13806d4-4f06-4df5-8280-549978c6d0ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:28.927423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13806d4-4f06-4df5-8280-549978c6d0ed None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:28.939112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6208a409-d135-4dbc-9e32-174cf7845b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:28.939781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208a409-d135-4dbc-9e32-174cf7845b39 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.940072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208a409-d135-4dbc-9e32-174cf7845b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.940352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208a409-d135-4dbc-9e32-174cf7845b39 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.941109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208a409-d135-4dbc-9e32-174cf7845b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:28.941423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208a409-d135-4dbc-9e32-174cf7845b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:28.942788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6208a409-d135-4dbc-9e32-174cf7845b39 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-e13806d4-4f06-4df5-8280-549978c6d0ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:28.945355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:28.947431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:28.949610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:28.949610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:28.949610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:28.949610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:28.949610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 365/765] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:28.949610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ef93-3869-4d2a-89dc-8bd23d52bf9a None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.949610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ef93-3869-4d2a-89dc-8bd23d52bf9a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.949610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ef93-3869-4d2a-89dc-8bd23d52bf9a None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.955426 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 366/766] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:28.955686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 401/767] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:28.956303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585ef93-3869-4d2a-89dc-8bd23d52bf9a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:28.961494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5b222-2d12-4f3a-8f20-6c4cbf561c43 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:28.961792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5b222-2d12-4f3a-8f20-6c4cbf561c43 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:28.962077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec5b222-2d12-4f3a-8f20-6c4cbf561c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:28.979565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1ec5b222-2d12-4f3a-8f20-6c4cbf561c43 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:28.980877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1ec5b222-2d12-4f3a-8f20-6c4cbf561c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:29.032710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:29.055375 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 402/768] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:29.056314 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5b222-2d12-4f3a-8f20-6c4cbf561c43 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:29.088377 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:29.096926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585ef93-3869-4d2a-89dc-8bd23d52bf9a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:29.146344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f93ce-0629-43ca-8734-2b1172ed5504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:29.147502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f93ce-0629-43ca-8734-2b1172ed5504 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:29.147788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f93ce-0629-43ca-8734-2b1172ed5504 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:29.148327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f93ce-0629-43ca-8734-2b1172ed5504 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:29.149101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f93ce-0629-43ca-8734-2b1172ed5504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:29.149649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f93ce-0629-43ca-8734-2b1172ed5504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:29.151412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f93ce-0629-43ca-8734-2b1172ed5504 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:29.168914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 367/769] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:29.169874 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f93ce-0629-43ca-8734-2b1172ed5504 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:29.174898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec5b222-2d12-4f3a-8f20-6c4cbf561c43 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:29.211555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bd406-996e-4c6b-952e-55f82e48465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:29.212269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bd406-996e-4c6b-952e-55f82e48465a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:29.212554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bd406-996e-4c6b-952e-55f82e48465a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:29.212855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bd406-996e-4c6b-952e-55f82e48465a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:29.220002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bd406-996e-4c6b-952e-55f82e48465a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:29.221493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bd406-996e-4c6b-952e-55f82e48465a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2e4bd406-996e-4c6b-952e-55f82e48465a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:29.236772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:29.238986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:29.241198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:29.241198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:29.241198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:29.241198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:29.241198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:29.241198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:29.241198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:29.241198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:29.241198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 403/770] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:29.241198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bd406-996e-4c6b-952e-55f82e48465a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:29.281726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f93ce-0629-43ca-8734-2b1172ed5504 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:29.306842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:29.319088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e298-95ab-4ba0-8250-56ff749671ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:29.319997 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e298-95ab-4ba0-8250-56ff749671ee None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:29.320357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e298-95ab-4ba0-8250-56ff749671ee None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:29.320736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e298-95ab-4ba0-8250-56ff749671ee None admin] PATH_INFO: `/v3/services/network` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:29.321495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8e298-95ab-4ba0-8250-56ff749671ee None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:29.323068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b8e298-95ab-4ba0-8250-56ff749671ee None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-32b8e298-95ab-4ba0-8250-56ff749671ee None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:29.336147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:29.338208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:42:29.340427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:29.342281 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:29.352721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 368/771] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:29 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:29.352721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e298-95ab-4ba0-8250-56ff749671ee None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:29.353163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bd406-996e-4c6b-952e-55f82e48465a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:29.391362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e5690-57d6-4c92-a971-6f69252b9c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:29.392303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e5690-57d6-4c92-a971-6f69252b9c36 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:29.392582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e5690-57d6-4c92-a971-6f69252b9c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:29.392936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e5690-57d6-4c92-a971-6f69252b9c36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:29.400494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e5690-57d6-4c92-a971-6f69252b9c36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:29.402074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e5690-57d6-4c92-a971-6f69252b9c36 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-ba9e5690-57d6-4c92-a971-6f69252b9c36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:29.417773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:29.419853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:29.422147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:29.422147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:29.422147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:29.422147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:29.422147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:29.422147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:29.422147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:29.422147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:29.428712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 404/772] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:29.428712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e5690-57d6-4c92-a971-6f69252b9c36 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:29.454660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e298-95ab-4ba0-8250-56ff749671ee None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:29.490609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6903276-08cb-44fc-bd6a-511f10b28f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:29.491272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6903276-08cb-44fc-bd6a-511f10b28f36 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:29.491562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6903276-08cb-44fc-bd6a-511f10b28f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:29.491956 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6903276-08cb-44fc-bd6a-511f10b28f36 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:29.492803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6903276-08cb-44fc-bd6a-511f10b28f36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:29.493123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6903276-08cb-44fc-bd6a-511f10b28f36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:29.494563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6903276-08cb-44fc-bd6a-511f10b28f36 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:29.512876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 369/773] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:29.513617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6903276-08cb-44fc-bd6a-511f10b28f36 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:29.533863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e5690-57d6-4c92-a971-6f69252b9c36 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:29.565769 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:29.572341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55473bc3-85c6-4884-8ebd-691d80346882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:29.573037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473bc3-85c6-4884-8ebd-691d80346882 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:29.573413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473bc3-85c6-4884-8ebd-691d80346882 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:29.573696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473bc3-85c6-4884-8ebd-691d80346882 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:29.574507 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473bc3-85c6-4884-8ebd-691d80346882 None admin] RBAC: Adding query filter params (name=network) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:29.574818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473bc3-85c6-4884-8ebd-691d80346882 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:29.576187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473bc3-85c6-4884-8ebd-691d80346882 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:29.582158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 405/774] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:29.582961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55473bc3-85c6-4884-8ebd-691d80346882 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:29.599662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:29.626319 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6903276-08cb-44fc-bd6a-511f10b28f36 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:29.668442 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e04a71-a1df-463c-a30d-c2b7b9022b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:29.669142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e04a71-a1df-463c-a30d-c2b7b9022b3e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:29.669436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e04a71-a1df-463c-a30d-c2b7b9022b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:29.669720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e04a71-a1df-463c-a30d-c2b7b9022b3e None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:29.670586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e04a71-a1df-463c-a30d-c2b7b9022b3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:29.670904 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e04a71-a1df-463c-a30d-c2b7b9022b3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:29.672338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e04a71-a1df-463c-a30d-c2b7b9022b3e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:29.684183 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 370/775] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:29.685002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e04a71-a1df-463c-a30d-c2b7b9022b3e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:29.698813 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55473bc3-85c6-4884-8ebd-691d80346882 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:29.735995 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ee7a6-b28f-44f0-82a2-d31581320b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:29.736665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ee7a6-b28f-44f0-82a2-d31581320b1d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:29.736976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ee7a6-b28f-44f0-82a2-d31581320b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:29.737461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ee7a6-b28f-44f0-82a2-d31581320b1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:29.738090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ee7a6-b28f-44f0-82a2-d31581320b1d None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=646af794697941d6a46d76086e7e8ba9) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:29.738498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ee7a6-b28f-44f0-82a2-d31581320b1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:29.739934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ee7a6-b28f-44f0-82a2-d31581320b1d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:29.745779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 406/776] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:29 2022] GET /identity/v3/role_assignments?user.id=646af794697941d6a46d76086e7e8ba9&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:29.750839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef112fa-a34f-4b5c-a320-566b592fa89d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:29.752031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef112fa-a34f-4b5c-a320-566b592fa89d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:29.752700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef112fa-a34f-4b5c-a320-566b592fa89d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:29.759623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 407/777] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:29 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:29.759623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef112fa-a34f-4b5c-a320-566b592fa89d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:29.808958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e04a71-a1df-463c-a30d-c2b7b9022b3e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:29.820522 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:29.851123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dc06c-ff2a-4550-86b4-9bd1889220e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:29.851804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dc06c-ff2a-4550-86b4-9bd1889220e5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:29.852126 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dc06c-ff2a-4550-86b4-9bd1889220e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:29.852403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dc06c-ff2a-4550-86b4-9bd1889220e5 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:29.853204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dc06c-ff2a-4550-86b4-9bd1889220e5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:29.853514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dc06c-ff2a-4550-86b4-9bd1889220e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:29.854970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dc06c-ff2a-4550-86b4-9bd1889220e5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:29.861448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 371/778] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:29.862843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0fa01-3fa0-4220-b3fb-4a0c15359843 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:29.863133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0fa01-3fa0-4220-b3fb-4a0c15359843 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:29.863465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0fa01-3fa0-4220-b3fb-4a0c15359843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:29.882923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ffc0fa01-3fa0-4220-b3fb-4a0c15359843 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:29.884204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ffc0fa01-3fa0-4220-b3fb-4a0c15359843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:29.891981 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:29.922919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef112fa-a34f-4b5c-a320-566b592fa89d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:29.966068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27a842-6691-4372-8e3b-95104a7e39a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:29.967374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27a842-6691-4372-8e3b-95104a7e39a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:29.967872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27a842-6691-4372-8e3b-95104a7e39a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:29.968376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27a842-6691-4372-8e3b-95104a7e39a4 None admin] PATH_INFO: `/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/36c18bbd22f74efbae7aab1cdd3f6403/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:29.970652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27a842-6691-4372-8e3b-95104a7e39a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=19c2f661cc934aa7869e9b3d4efa875e, user_id=36c18bbd22f74efbae7aab1cdd3f6403, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:29.971691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 372/779] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:29.972556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27a842-6691-4372-8e3b-95104a7e39a4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:29.972690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0fa01-3fa0-4220-b3fb-4a0c15359843 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:29.987455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 408/780] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:42:29 2022] PUT /identity/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/36c18bbd22f74efbae7aab1cdd3f6403/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:30.026462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27a842-6691-4372-8e3b-95104a7e39a4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:30.106552 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0fa01-3fa0-4220-b3fb-4a0c15359843 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:30.108217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:30.147986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f07afd-8e27-4220-b67a-02bdbc9351df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:30.149040 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f07afd-8e27-4220-b67a-02bdbc9351df None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:30.149040 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f07afd-8e27-4220-b67a-02bdbc9351df None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:30.149677 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f07afd-8e27-4220-b67a-02bdbc9351df None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:30.150129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f07afd-8e27-4220-b67a-02bdbc9351df None admin] RBAC: Adding query filter params (interface=public, service_id=2320e542f36f4fbd9ea22094e92d225c, region_id=RegionOne) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:30.150446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f07afd-8e27-4220-b67a-02bdbc9351df None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:30.151827 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f07afd-8e27-4220-b67a-02bdbc9351df None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:30.165236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 373/781] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 23 11:42:29 2022] GET /identity/v3/endpoints?service_id=2320e542f36f4fbd9ea22094e92d225c&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:30.166560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f07afd-8e27-4220-b67a-02bdbc9351df None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:30.214448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27a842-6691-4372-8e3b-95104a7e39a4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:30.266460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f85a47-75fb-4eee-98f5-12ddc69defec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:30.270161 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f85a47-75fb-4eee-98f5-12ddc69defec None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:30.270448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f85a47-75fb-4eee-98f5-12ddc69defec None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:30.270728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f85a47-75fb-4eee-98f5-12ddc69defec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:30.290567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:30.304269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f85a47-75fb-4eee-98f5-12ddc69defec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-83f85a47-75fb-4eee-98f5-12ddc69defec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:30.313378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:30.314362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:30.315223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 409/782] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:42:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:30.320711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f85a47-75fb-4eee-98f5-12ddc69defec None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:30.322293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f07afd-8e27-4220-b67a-02bdbc9351df None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:30.371214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb6364-e04b-48ac-b343-0a9388f63674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:30.372947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb6364-e04b-48ac-b343-0a9388f63674 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:30.374066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb6364-e04b-48ac-b343-0a9388f63674 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:30.375078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb6364-e04b-48ac-b343-0a9388f63674 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:30.376400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb6364-e04b-48ac-b343-0a9388f63674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:30.378350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb6364-e04b-48ac-b343-0a9388f63674 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:30.386369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 374/783] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 23 11:42:30 2022] GET /identity/v3/services => generated 772 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:30.454657 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f85a47-75fb-4eee-98f5-12ddc69defec None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:30.501230 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036e77b-affb-4bfb-942b-874d50382bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:30.502304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036e77b-affb-4bfb-942b-874d50382bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:30.502805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036e77b-affb-4bfb-942b-874d50382bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:30.503302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036e77b-affb-4bfb-942b-874d50382bc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:30.504337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036e77b-affb-4bfb-942b-874d50382bc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:30.504963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036e77b-affb-4bfb-942b-874d50382bc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:30.506651 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036e77b-affb-4bfb-942b-874d50382bc3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:30.521520 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 410/784] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:30.526920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb6364-e04b-48ac-b343-0a9388f63674 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:30.566804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:30.650071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb6364-e04b-48ac-b343-0a9388f63674 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:30.691258 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11085bae-b3db-4d15-89a2-8907e41d3b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:30.692434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085bae-b3db-4d15-89a2-8907e41d3b51 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:30.692952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085bae-b3db-4d15-89a2-8907e41d3b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:30.693461 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11085bae-b3db-4d15-89a2-8907e41d3b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:30.706327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11085bae-b3db-4d15-89a2-8907e41d3b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:30.708103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11085bae-b3db-4d15-89a2-8907e41d3b51 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-11085bae-b3db-4d15-89a2-8907e41d3b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:30.726399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:30.727327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:30.728221 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:30.728221 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:30.728221 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:30.728221 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:30.730460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 375/785] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:30.735782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036e77b-affb-4bfb-942b-874d50382bc3 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:30.824427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:30.852900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f147e3-0b1e-4f4c-98c2-e0bec2675702 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:30.853411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f147e3-0b1e-4f4c-98c2-e0bec2675702 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:30.853800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f147e3-0b1e-4f4c-98c2-e0bec2675702 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:30.855546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 376/786] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:42:30.860645 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f02b0-c9ba-4b77-880f-62d3321e150c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:30.861097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f02b0-c9ba-4b77-880f-62d3321e150c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:30.861481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f02b0-c9ba-4b77-880f-62d3321e150c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:30.884892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036e77b-affb-4bfb-942b-874d50382bc3 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:30.897731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e41f02b0-c9ba-4b77-880f-62d3321e150c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:30.898971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e41f02b0-c9ba-4b77-880f-62d3321e150c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:30.936744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2bb4d-a66a-4263-ba84-0286ed8ebac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:30.937423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2bb4d-a66a-4263-ba84-0286ed8ebac7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:30.937712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2bb4d-a66a-4263-ba84-0286ed8ebac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:30.937993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2bb4d-a66a-4263-ba84-0286ed8ebac7 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:30.938726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2bb4d-a66a-4263-ba84-0286ed8ebac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:30.939040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2bb4d-a66a-4263-ba84-0286ed8ebac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:30.940442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e2bb4d-a66a-4263-ba84-0286ed8ebac7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:30.961273 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 411/787] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:30.973782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2bb4d-a66a-4263-ba84-0286ed8ebac7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:31.011859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 377/788] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:31.047720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f02b0-c9ba-4b77-880f-62d3321e150c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:31.073083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:31.151335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2bb4d-a66a-4263-ba84-0286ed8ebac7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:31.193558 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:31.208247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af7695-2419-4eb8-8ace-28d7e07b9635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:31.209112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af7695-2419-4eb8-8ace-28d7e07b9635 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:31.209404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af7695-2419-4eb8-8ace-28d7e07b9635 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:31.209680 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af7695-2419-4eb8-8ace-28d7e07b9635 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:31.217185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af7695-2419-4eb8-8ace-28d7e07b9635 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:31.219145 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af7695-2419-4eb8-8ace-28d7e07b9635 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-77af7695-2419-4eb8-8ace-28d7e07b9635 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:31.238850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:31.239883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:31.241635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:31.241635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:31.241635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:31.241635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:31.241635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:31.241635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:31.241635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:31.241635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:31.241635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:42:31.241635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:31.241635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 412/789] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:42:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:31.242240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f02b0-c9ba-4b77-880f-62d3321e150c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:31.245934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af7695-2419-4eb8-8ace-28d7e07b9635 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:31.304784 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd555735-d232-42e5-8f6f-dc219f93d6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:31.305668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd555735-d232-42e5-8f6f-dc219f93d6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:31.305957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd555735-d232-42e5-8f6f-dc219f93d6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:31.306235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd555735-d232-42e5-8f6f-dc219f93d6c0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:31.306897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd555735-d232-42e5-8f6f-dc219f93d6c0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:31.308266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd555735-d232-42e5-8f6f-dc219f93d6c0 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-fd555735-d232-42e5-8f6f-dc219f93d6c0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:31.340948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:31.343187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:31.345333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 378/790] 10.222.0.67 () {56 vars in 1129 bytes} [Thu Jun 23 11:42:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:31.348719 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af7695-2419-4eb8-8ace-28d7e07b9635 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:31.357754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd555735-d232-42e5-8f6f-dc219f93d6c0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:31.385616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756363f-3646-4fa0-975e-e82500471168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:31.386545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756363f-3646-4fa0-975e-e82500471168 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:31.386974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756363f-3646-4fa0-975e-e82500471168 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:31.387393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756363f-3646-4fa0-975e-e82500471168 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:31.388463 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756363f-3646-4fa0-975e-e82500471168 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:31.389015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756363f-3646-4fa0-975e-e82500471168 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:31.391216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756363f-3646-4fa0-975e-e82500471168 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:31.410389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 413/791] 10.222.0.67 () {56 vars in 1135 bytes} [Thu Jun 23 11:42:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:31.415999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756363f-3646-4fa0-975e-e82500471168 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:31.459468 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:31.527453 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756363f-3646-4fa0-975e-e82500471168 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:31.565794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4501b9c-241f-4335-80c9-79f4e40662f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:31.566840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4501b9c-241f-4335-80c9-79f4e40662f4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:31.567128 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4501b9c-241f-4335-80c9-79f4e40662f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:31.567450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4501b9c-241f-4335-80c9-79f4e40662f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:31.579446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4501b9c-241f-4335-80c9-79f4e40662f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:31.581241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4501b9c-241f-4335-80c9-79f4e40662f4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-d4501b9c-241f-4335-80c9-79f4e40662f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:31.602208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:31.603178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:31.604079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:31.604079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:31.604079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:31.604079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:31.604079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 414/792] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:31.609948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4501b9c-241f-4335-80c9-79f4e40662f4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:31.651219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:31.720142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:31.725224 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4501b9c-241f-4335-80c9-79f4e40662f4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:31.754672 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd555735-d232-42e5-8f6f-dc219f93d6c0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:31.763420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac3536-64c2-4a0e-8906-70227624f315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:31.764133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac3536-64c2-4a0e-8906-70227624f315 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:31.764421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac3536-64c2-4a0e-8906-70227624f315 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:31.764737 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac3536-64c2-4a0e-8906-70227624f315 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:31.765681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac3536-64c2-4a0e-8906-70227624f315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:31.766057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac3536-64c2-4a0e-8906-70227624f315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:31.767494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac3536-64c2-4a0e-8906-70227624f315 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:31.781288 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 415/793] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:31.787543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac3536-64c2-4a0e-8906-70227624f315 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:31.872726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5b03c-f4bc-4326-a4e8-45302ee2d49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:31.873982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5b03c-f4bc-4326-a4e8-45302ee2d49a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:31.874344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5b03c-f4bc-4326-a4e8-45302ee2d49a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:31.874671 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5b03c-f4bc-4326-a4e8-45302ee2d49a None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:31.875418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5b03c-f4bc-4326-a4e8-45302ee2d49a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:31.875805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5b03c-f4bc-4326-a4e8-45302ee2d49a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:31.877216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5b03c-f4bc-4326-a4e8-45302ee2d49a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:31.888097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 379/794] 10.222.0.67 () {56 vars in 1124 bytes} [Thu Jun 23 11:42:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 531 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:31.894155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5b03c-f4bc-4326-a4e8-45302ee2d49a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:31.898373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac3536-64c2-4a0e-8906-70227624f315 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:31.963521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0516a-d3fd-4510-bc1e-5d704cd3ffaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:31.964299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0516a-d3fd-4510-bc1e-5d704cd3ffaa None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:31.964585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0516a-d3fd-4510-bc1e-5d704cd3ffaa None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:31.965022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0516a-d3fd-4510-bc1e-5d704cd3ffaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:31.977315 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0516a-d3fd-4510-bc1e-5d704cd3ffaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:31.980773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d0516a-d3fd-4510-bc1e-5d704cd3ffaa None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-13d0516a-d3fd-4510-bc1e-5d704cd3ffaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:32.007573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:32.009714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:32.011812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:32.011812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:32.011812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:32.011812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:32.011812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:32.011812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:32.011812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:32.011812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:32.018610 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 416/795] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:32.021990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0516a-d3fd-4510-bc1e-5d704cd3ffaa None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:32.044296 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:32.070869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5b03c-f4bc-4326-a4e8-45302ee2d49a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:32.114077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838981b9-baa5-4073-9b71-3d24d83f049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:32.114745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838981b9-baa5-4073-9b71-3d24d83f049f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:32.115031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838981b9-baa5-4073-9b71-3d24d83f049f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:32.115304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838981b9-baa5-4073-9b71-3d24d83f049f None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:32.116013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838981b9-baa5-4073-9b71-3d24d83f049f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:32.116319 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838981b9-baa5-4073-9b71-3d24d83f049f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:32.117742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838981b9-baa5-4073-9b71-3d24d83f049f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:32.123365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 380/796] 10.222.0.67 () {56 vars in 1124 bytes} [Thu Jun 23 11:42:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:32.124752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af01b11-8d30-4722-a1e3-c5d194d6a86b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:32.125043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af01b11-8d30-4722-a1e3-c5d194d6a86b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:32.125323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af01b11-8d30-4722-a1e3-c5d194d6a86b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:32.130896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 381/797] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:32.132207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cb2b6-9ea3-4253-b457-260e761c55f3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:32.132497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cb2b6-9ea3-4253-b457-260e761c55f3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:32.132826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cb2b6-9ea3-4253-b457-260e761c55f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:32.138992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:32.150268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d19cb2b6-9ea3-4253-b457-260e761c55f3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:32.151525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d19cb2b6-9ea3-4253-b457-260e761c55f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:32.153570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0516a-d3fd-4510-bc1e-5d704cd3ffaa None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:32.193787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbcb12-a3e2-415e-b3e5-7f5fe7bc7aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:32.194468 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbcb12-a3e2-415e-b3e5-7f5fe7bc7aad None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:32.194758 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbcb12-a3e2-415e-b3e5-7f5fe7bc7aad None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:32.195042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fbcb12-a3e2-415e-b3e5-7f5fe7bc7aad None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:32.195854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbcb12-a3e2-415e-b3e5-7f5fe7bc7aad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:32.196287 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbcb12-a3e2-415e-b3e5-7f5fe7bc7aad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:32.197811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fbcb12-a3e2-415e-b3e5-7f5fe7bc7aad None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:32.211509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 417/798] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:32.212362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbcb12-a3e2-415e-b3e5-7f5fe7bc7aad None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:32.237004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 382/799] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:42:32.238045 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cb2b6-9ea3-4253-b457-260e761c55f3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:32.297365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:32.336052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fbcb12-a3e2-415e-b3e5-7f5fe7bc7aad None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:32.378041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636501eb-73c0-49e9-8cac-b308dd7c8028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:32.378723 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636501eb-73c0-49e9-8cac-b308dd7c8028 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:32.379012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636501eb-73c0-49e9-8cac-b308dd7c8028 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:32.379182 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cb2b6-9ea3-4253-b457-260e761c55f3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:32.379636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636501eb-73c0-49e9-8cac-b308dd7c8028 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:32.380047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636501eb-73c0-49e9-8cac-b308dd7c8028 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:32.381618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636501eb-73c0-49e9-8cac-b308dd7c8028 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:32.412794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 418/800] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 23 11:42:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:32.414005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636501eb-73c0-49e9-8cac-b308dd7c8028 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:32.442869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:32.452612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d0a60-27ad-4cc7-b3f7-db9193299ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:32.454358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d0a60-27ad-4cc7-b3f7-db9193299ac6 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:32.454647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d0a60-27ad-4cc7-b3f7-db9193299ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:32.454931 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d0a60-27ad-4cc7-b3f7-db9193299ac6 None admin] PATH_INFO: `/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/eb9b9af5e0b44e34837e1cdb5ff6d33e/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:32.460655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d0a60-27ad-4cc7-b3f7-db9193299ac6 None admin] RBAC: Authorizing `identity:create_grant(project_id=19c2f661cc934aa7869e9b3d4efa875e, user_id=eb9b9af5e0b44e34837e1cdb5ff6d33e, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:32.462474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d0a60-27ad-4cc7-b3f7-db9193299ac6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:32.492735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 383/801] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:42:32 2022] PUT /identity/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/eb9b9af5e0b44e34837e1cdb5ff6d33e/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:32.594176 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636501eb-73c0-49e9-8cac-b308dd7c8028 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:32.631558 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:32.647162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d4f31-8751-4f91-962b-7b44e55534f4 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:32.647683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ffdab-a21a-4436-9a1b-5c17ff63a694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:32.648065 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d4f31-8751-4f91-962b-7b44e55534f4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:32.648500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ffdab-a21a-4436-9a1b-5c17ff63a694 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:32.648816 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ffdab-a21a-4436-9a1b-5c17ff63a694 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:32.648920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d4f31-8751-4f91-962b-7b44e55534f4 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:32.649087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ffdab-a21a-4436-9a1b-5c17ff63a694 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:32.651271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 384/802] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:42:32.668425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ffdab-a21a-4436-9a1b-5c17ff63a694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:32.669836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3c606-dedc-407b-b59e-ac275bebbf10 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:32.670426 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3c606-dedc-407b-b59e-ac275bebbf10 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:32.670905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3c606-dedc-407b-b59e-ac275bebbf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-398ffdab-a21a-4436-9a1b-5c17ff63a694 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:32.676626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:32.677716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:32.678557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:32.678557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:32.678557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:32.678557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:32.678557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:32.678557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:32.678557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:32.678557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:32.678557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:32.678557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:32.678557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:32.678557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:32.685182 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 419/803] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:32.686277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ffdab-a21a-4436-9a1b-5c17ff63a694 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:32.694614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3ee3c606-dedc-407b-b59e-ac275bebbf10 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:32.696211 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3ee3c606-dedc-407b-b59e-ac275bebbf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:32.794668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 385/804] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:32.815712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ffdab-a21a-4436-9a1b-5c17ff63a694 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:32.832407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3c606-dedc-407b-b59e-ac275bebbf10 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:32.869288 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd76d42-6ae1-4716-ba65-9e4f5635756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:32.869963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd76d42-6ae1-4716-ba65-9e4f5635756e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:32.870246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd76d42-6ae1-4716-ba65-9e4f5635756e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:32.870518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd76d42-6ae1-4716-ba65-9e4f5635756e None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:32.871272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd76d42-6ae1-4716-ba65-9e4f5635756e None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:32.871579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd76d42-6ae1-4716-ba65-9e4f5635756e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:32.872970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd76d42-6ae1-4716-ba65-9e4f5635756e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:32.886291 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 420/805] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:32.892746 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd76d42-6ae1-4716-ba65-9e4f5635756e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:33.001374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:33.027694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3c606-dedc-407b-b59e-ac275bebbf10 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:33.080563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:33.089063 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02913371-f366-4175-813a-b29b0d7bac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:33.089912 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02913371-f366-4175-813a-b29b0d7bac31 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:33.090195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02913371-f366-4175-813a-b29b0d7bac31 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:33.090494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02913371-f366-4175-813a-b29b0d7bac31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:33.095106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd76d42-6ae1-4716-ba65-9e4f5635756e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:33.114385 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02913371-f366-4175-813a-b29b0d7bac31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-02913371-f366-4175-813a-b29b0d7bac31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:33.126770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:33.127831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:33.128710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:33.128710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:33.128710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:33.128710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:33.128710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:33.128710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:33.128710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:33.128710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:33.128710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:33.128710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:33.128710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:42:33.128710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:33.131293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 386/806] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:42:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:33.133426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86ba53-cf43-4388-84a8-2dda696f12c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:33.134490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86ba53-cf43-4388-84a8-2dda696f12c9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:33.134989 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86ba53-cf43-4388-84a8-2dda696f12c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:33.135469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af86ba53-cf43-4388-84a8-2dda696f12c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:33.141145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02913371-f366-4175-813a-b29b0d7bac31 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:33.148567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86ba53-cf43-4388-84a8-2dda696f12c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:33.150325 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af86ba53-cf43-4388-84a8-2dda696f12c9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-af86ba53-cf43-4388-84a8-2dda696f12c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:33.170335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:33.171331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:33.172233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:33.172233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:33.172233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:33.172233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:33.174571 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 421/807] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:33.180187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86ba53-cf43-4388-84a8-2dda696f12c9 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:33.382267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02913371-f366-4175-813a-b29b0d7bac31 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:33.407120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:33.411456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af86ba53-cf43-4388-84a8-2dda696f12c9 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:33.426820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a308e97-ecdd-4687-a408-c41e7264470f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:33.427490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a308e97-ecdd-4687-a408-c41e7264470f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:33.427774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a308e97-ecdd-4687-a408-c41e7264470f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:33.428050 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a308e97-ecdd-4687-a408-c41e7264470f None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:33.429101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a308e97-ecdd-4687-a408-c41e7264470f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:33.429101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a308e97-ecdd-4687-a408-c41e7264470f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:33.430495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a308e97-ecdd-4687-a408-c41e7264470f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:33.444774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 387/808] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:33.444774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f70ee-8117-412c-a31c-922114e9db7a None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:33.444774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f70ee-8117-412c-a31c-922114e9db7a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:33.444774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f70ee-8117-412c-a31c-922114e9db7a None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:33.447944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 388/809] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:33.448955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f70ee-8117-412c-a31c-922114e9db7a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:33.458568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22467990-3042-45f0-bd69-7f072506d23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:33.459249 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22467990-3042-45f0-bd69-7f072506d23b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:33.459547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22467990-3042-45f0-bd69-7f072506d23b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:33.459829 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22467990-3042-45f0-bd69-7f072506d23b None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:33.460566 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22467990-3042-45f0-bd69-7f072506d23b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:33.460926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22467990-3042-45f0-bd69-7f072506d23b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:33.462437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22467990-3042-45f0-bd69-7f072506d23b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:33.477300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 422/810] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:33.478928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd542b7-bcc3-46a5-b757-0aa39ed8a054 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:33.479385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd542b7-bcc3-46a5-b757-0aa39ed8a054 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:33.479676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd542b7-bcc3-46a5-b757-0aa39ed8a054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:33.498357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:33.501098 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-dbd542b7-bcc3-46a5-b757-0aa39ed8a054 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:33.502346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-dbd542b7-bcc3-46a5-b757-0aa39ed8a054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:33.571856 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f70ee-8117-412c-a31c-922114e9db7a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:33.596952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 423/811] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:33.598453 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd542b7-bcc3-46a5-b757-0aa39ed8a054 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:33.614984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d76e9-4db8-4a45-9952-5b70c1e6f7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:33.615790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d76e9-4db8-4a45-9952-5b70c1e6f7bd None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:33.616140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d76e9-4db8-4a45-9952-5b70c1e6f7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:33.616423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d76e9-4db8-4a45-9952-5b70c1e6f7bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:33.630899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d76e9-4db8-4a45-9952-5b70c1e6f7bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:33.632720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4d76e9-4db8-4a45-9952-5b70c1e6f7bd None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-bd4d76e9-4db8-4a45-9952-5b70c1e6f7bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:33.665741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:33.666794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:33.667741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:33.667741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:33.667741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:33.667741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:33.674992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 389/812] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:33.674992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d76e9-4db8-4a45-9952-5b70c1e6f7bd None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:33.722241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:33.735447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd542b7-bcc3-46a5-b757-0aa39ed8a054 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:33.776205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73fdb1-f2ea-44ff-bbbd-17fc069ece5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:33.777088 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73fdb1-f2ea-44ff-bbbd-17fc069ece5a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:33.777389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73fdb1-f2ea-44ff-bbbd-17fc069ece5a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:33.777672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73fdb1-f2ea-44ff-bbbd-17fc069ece5a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:33.786795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73fdb1-f2ea-44ff-bbbd-17fc069ece5a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:33.788079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73fdb1-f2ea-44ff-bbbd-17fc069ece5a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3f73fdb1-f2ea-44ff-bbbd-17fc069ece5a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:33.804516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:33.805560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:33.806493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:33.806493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:33.806493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:33.806493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:33.806493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:33.806493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:33.806493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:33.806493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:33.806493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:42:33.806493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:33.806493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 424/813] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 23 11:42:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:33.807575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73fdb1-f2ea-44ff-bbbd-17fc069ece5a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:33.813573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:33.822430 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d76e9-4db8-4a45-9952-5b70c1e6f7bd None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:33.857705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82e3dd-0b7c-47ed-8259-0bbbaa4b0d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:33.858977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82e3dd-0b7c-47ed-8259-0bbbaa4b0d9b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:33.859480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82e3dd-0b7c-47ed-8259-0bbbaa4b0d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:33.859961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82e3dd-0b7c-47ed-8259-0bbbaa4b0d9b None admin] PATH_INFO: `/v3/services/network` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:33.860913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82e3dd-0b7c-47ed-8259-0bbbaa4b0d9b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:33.862662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a82e3dd-0b7c-47ed-8259-0bbbaa4b0d9b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-8a82e3dd-0b7c-47ed-8259-0bbbaa4b0d9b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:33.876518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:33.877604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 11:42:33.878422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:33.881331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 390/814] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:33 2022] GET /identity/v3/services/network => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:33.882372 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82e3dd-0b7c-47ed-8259-0bbbaa4b0d9b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:33.923819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73fdb1-f2ea-44ff-bbbd-17fc069ece5a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:33.961384 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea36995-8b3b-40f6-80c2-f795c6882908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:33.962057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea36995-8b3b-40f6-80c2-f795c6882908 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:33.962343 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea36995-8b3b-40f6-80c2-f795c6882908 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:33.962625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea36995-8b3b-40f6-80c2-f795c6882908 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:33.963640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea36995-8b3b-40f6-80c2-f795c6882908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:33.963752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea36995-8b3b-40f6-80c2-f795c6882908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:33.965323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea36995-8b3b-40f6-80c2-f795c6882908 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:33.978895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 425/815] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:33.979722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea36995-8b3b-40f6-80c2-f795c6882908 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:33.987647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a82e3dd-0b7c-47ed-8259-0bbbaa4b0d9b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:33.993029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:34.024632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308cc54-98f6-4a77-a376-9b47b4e9eeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:34.025702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308cc54-98f6-4a77-a376-9b47b4e9eeaf None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:34.026210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308cc54-98f6-4a77-a376-9b47b4e9eeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:34.026737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b308cc54-98f6-4a77-a376-9b47b4e9eeaf None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:34.027770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b308cc54-98f6-4a77-a376-9b47b4e9eeaf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:34.028377 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b308cc54-98f6-4a77-a376-9b47b4e9eeaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:34.030720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b308cc54-98f6-4a77-a376-9b47b4e9eeaf None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:34.050023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 391/816] 10.222.0.67 () {56 vars in 1149 bytes} [Thu Jun 23 11:42:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:34.051177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308cc54-98f6-4a77-a376-9b47b4e9eeaf None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:34.069648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:34.099432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea36995-8b3b-40f6-80c2-f795c6882908 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:34.135761 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f63d9-1130-42e9-b84f-29f3182c1654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:34.136429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f63d9-1130-42e9-b84f-29f3182c1654 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:34.136743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f63d9-1130-42e9-b84f-29f3182c1654 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:34.137082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f63d9-1130-42e9-b84f-29f3182c1654 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:34.137961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f63d9-1130-42e9-b84f-29f3182c1654 None admin] RBAC: Adding query filter params (name=network) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:34.138307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f63d9-1130-42e9-b84f-29f3182c1654 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:34.140313 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f63d9-1130-42e9-b84f-29f3182c1654 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:34.145393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 426/817] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:34.146440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f63d9-1130-42e9-b84f-29f3182c1654 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:34.159595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b308cc54-98f6-4a77-a376-9b47b4e9eeaf None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:34.195157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470ad70-a538-481c-a685-8e5d0d2c0cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:34.196388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470ad70-a538-481c-a685-8e5d0d2c0cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:34.196898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470ad70-a538-481c-a685-8e5d0d2c0cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:34.197434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470ad70-a538-481c-a685-8e5d0d2c0cd6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:34.205873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470ad70-a538-481c-a685-8e5d0d2c0cd6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:34.207834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470ad70-a538-481c-a685-8e5d0d2c0cd6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-e470ad70-a538-481c-a685-8e5d0d2c0cd6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:34.224267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:34.225268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:34.226205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:34.226205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:34.226205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:34.226205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:34.226205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:34.226205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:34.226205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:34.226205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:34.226205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:42:34.226205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:34.227978 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:34.230023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 392/818] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 23 11:42:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:34.231218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470ad70-a538-481c-a685-8e5d0d2c0cd6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:34.258242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f63d9-1130-42e9-b84f-29f3182c1654 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:34.294835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984eca6d-ef7f-4d7c-9d12-0d6f38caab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:34.296277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984eca6d-ef7f-4d7c-9d12-0d6f38caab87 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:34.296565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984eca6d-ef7f-4d7c-9d12-0d6f38caab87 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:34.296870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984eca6d-ef7f-4d7c-9d12-0d6f38caab87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:34.309373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984eca6d-ef7f-4d7c-9d12-0d6f38caab87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:34.310303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984eca6d-ef7f-4d7c-9d12-0d6f38caab87 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:34.317633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-984eca6d-ef7f-4d7c-9d12-0d6f38caab87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:34.331717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:34.332644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:34.333585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:34.333585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:34.333585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:34.333585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:34.333585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 427/819] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:34.333867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984eca6d-ef7f-4d7c-9d12-0d6f38caab87 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:34.344933 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470ad70-a538-481c-a685-8e5d0d2c0cd6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:34.382344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c243c-1c56-4132-b786-65f4523f50ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:34.383885 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c243c-1c56-4132-b786-65f4523f50ca None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:34.384399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c243c-1c56-4132-b786-65f4523f50ca None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:34.384952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c243c-1c56-4132-b786-65f4523f50ca None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:34.385886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c243c-1c56-4132-b786-65f4523f50ca None admin] RBAC: Adding query filter params (type=network) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:34.386395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c243c-1c56-4132-b786-65f4523f50ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:34.388127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c243c-1c56-4132-b786-65f4523f50ca None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:34.394145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 393/820] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:34.395391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c243c-1c56-4132-b786-65f4523f50ca None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:34.451478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984eca6d-ef7f-4d7c-9d12-0d6f38caab87 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:34.479629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:34.490888 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d69c1-dc5a-404a-8a2d-df7234d783e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:34.491600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d69c1-dc5a-404a-8a2d-df7234d783e9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:34.491890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d69c1-dc5a-404a-8a2d-df7234d783e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:34.492173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d69c1-dc5a-404a-8a2d-df7234d783e9 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:34.493068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d69c1-dc5a-404a-8a2d-df7234d783e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:34.493385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d69c1-dc5a-404a-8a2d-df7234d783e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:34.495185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d69c1-dc5a-404a-8a2d-df7234d783e9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:34.508344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c243c-1c56-4132-b786-65f4523f50ca None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:34.517772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 428/821] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 23 11:42:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:34.518680 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d69c1-dc5a-404a-8a2d-df7234d783e9 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:34.556316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d331a06-27a1-4ddf-81f9-e3e3501dae0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:34.557479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d331a06-27a1-4ddf-81f9-e3e3501dae0d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:34.557965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d331a06-27a1-4ddf-81f9-e3e3501dae0d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:34.558454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d331a06-27a1-4ddf-81f9-e3e3501dae0d None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:34.559370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d331a06-27a1-4ddf-81f9-e3e3501dae0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:34.559976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d331a06-27a1-4ddf-81f9-e3e3501dae0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:34.561736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d331a06-27a1-4ddf-81f9-e3e3501dae0d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:34.571925 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:34.581049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 394/822] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:34.582510 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d331a06-27a1-4ddf-81f9-e3e3501dae0d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:34.646081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d69c1-dc5a-404a-8a2d-df7234d783e9 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:34.683959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936f15b-2b36-4f45-9981-4264a6e1edd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:34.684699 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936f15b-2b36-4f45-9981-4264a6e1edd5 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:34.684991 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936f15b-2b36-4f45-9981-4264a6e1edd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:34.685266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936f15b-2b36-4f45-9981-4264a6e1edd5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:34.686176 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936f15b-2b36-4f45-9981-4264a6e1edd5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:34.687546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936f15b-2b36-4f45-9981-4264a6e1edd5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:34.709803 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 429/823] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 23 11:42:34 2022] POST /identity/v3/endpoints => generated 327 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:34.710727 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936f15b-2b36-4f45-9981-4264a6e1edd5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:34.714166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d331a06-27a1-4ddf-81f9-e3e3501dae0d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:34.760767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5e6c0-311c-4135-bd02-66ed20d4226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:34.761931 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5e6c0-311c-4135-bd02-66ed20d4226b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:34.762424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5e6c0-311c-4135-bd02-66ed20d4226b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:34.762903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5e6c0-311c-4135-bd02-66ed20d4226b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:34.764420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:34.836654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936f15b-2b36-4f45-9981-4264a6e1edd5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:34.837866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:34.846610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5e6c0-311c-4135-bd02-66ed20d4226b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:34.848081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5e6c0-311c-4135-bd02-66ed20d4226b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:34.879512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cde5a84-66bb-4823-98f4-68c76c2af741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:34.880320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde5a84-66bb-4823-98f4-68c76c2af741 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:34.880606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde5a84-66bb-4823-98f4-68c76c2af741 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:34.880926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cde5a84-66bb-4823-98f4-68c76c2af741 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:34.893037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde5a84-66bb-4823-98f4-68c76c2af741 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:34.894482 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cde5a84-66bb-4823-98f4-68c76c2af741 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3cde5a84-66bb-4823-98f4-68c76c2af741 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:34.911174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:34.912172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:34.913707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:34.913707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:34.913707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:34.913707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:34.913707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:34.913707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:34.913707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:34.913707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:34.913707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 430/824] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:34.914225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0f8d8-dd7a-424a-a698-cedd311f6963 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:34.914474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0f8d8-dd7a-424a-a698-cedd311f6963 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:34.914666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0f8d8-dd7a-424a-a698-cedd311f6963 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:34.920474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 431/825] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:34.920474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0f8d8-dd7a-424a-a698-cedd311f6963 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:34.991670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 395/826] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:34 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:42:34.991670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e555099-948a-4044-bd94-619f5c49610e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:34.991670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e555099-948a-4044-bd94-619f5c49610e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:34.991670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e555099-948a-4044-bd94-619f5c49610e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.008434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7e555099-948a-4044-bd94-619f5c49610e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:35.009692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7e555099-948a-4044-bd94-619f5c49610e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:35.043051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:35.047226 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0f8d8-dd7a-424a-a698-cedd311f6963 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:35.083473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 396/827] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:35.084457 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a17c6-0953-4521-8c26-a45e5203386d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:35.084793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d2e37-fa09-4a91-a986-965cae2eb0b8 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:35.085096 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d2e37-fa09-4a91-a986-965cae2eb0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:35.085301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a17c6-0953-4521-8c26-a45e5203386d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:35.085406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d2e37-fa09-4a91-a986-965cae2eb0b8 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.085585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a17c6-0953-4521-8c26-a45e5203386d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:35.085861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a17c6-0953-4521-8c26-a45e5203386d None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.086678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a17c6-0953-4521-8c26-a45e5203386d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:35.086993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a17c6-0953-4521-8c26-a45e5203386d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:35.088424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a17c6-0953-4521-8c26-a45e5203386d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:35.088793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 397/828] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:35.089752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d2e37-fa09-4a91-a986-965cae2eb0b8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:35.103821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 432/829] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:35.105163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69befe-0218-48cc-ac3f-39e02e0eaf9d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:35.105456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69befe-0218-48cc-ac3f-39e02e0eaf9d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:35.105738 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69befe-0218-48cc-ac3f-39e02e0eaf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.118991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:35.122338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-fd69befe-0218-48cc-ac3f-39e02e0eaf9d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:35.124225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fd69befe-0218-48cc-ac3f-39e02e0eaf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:35.190752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 433/830] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:35.191830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69befe-0218-48cc-ac3f-39e02e0eaf9d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:35.197083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d2e37-fa09-4a91-a986-965cae2eb0b8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:35.231546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcd14c-b466-4545-9efa-7953e4abc7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:35.232351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcd14c-b466-4545-9efa-7953e4abc7b6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:35.232665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcd14c-b466-4545-9efa-7953e4abc7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:35.232964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bcd14c-b466-4545-9efa-7953e4abc7b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.245764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcd14c-b466-4545-9efa-7953e4abc7b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:35.247146 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bcd14c-b466-4545-9efa-7953e4abc7b6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-10bcd14c-b466-4545-9efa-7953e4abc7b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:35.265603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:35.266748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:35.267705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:35.267705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:35.267705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:35.267705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:35.267705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 398/831] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:35.267705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcd14c-b466-4545-9efa-7953e4abc7b6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:35.287136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:35.296813 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69befe-0218-48cc-ac3f-39e02e0eaf9d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:35.331222 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c7a40-7f85-4e3f-88f7-c4408ea26e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:35.332016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c7a40-7f85-4e3f-88f7-c4408ea26e3f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:35.332302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c7a40-7f85-4e3f-88f7-c4408ea26e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:35.332602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c7a40-7f85-4e3f-88f7-c4408ea26e3f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.333289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c7a40-7f85-4e3f-88f7-c4408ea26e3f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:35.334680 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c7a40-7f85-4e3f-88f7-c4408ea26e3f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:35.343490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-962c7a40-7f85-4e3f-88f7-c4408ea26e3f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:35.346519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:35.347793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:35.349285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 434/832] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:42:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:35.350284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c7a40-7f85-4e3f-88f7-c4408ea26e3f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:35.371722 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bcd14c-b466-4545-9efa-7953e4abc7b6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:35.405568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1241f5-7c96-4c6a-8e79-576870d68ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:35.406455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1241f5-7c96-4c6a-8e79-576870d68ced None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:35.406740 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1241f5-7c96-4c6a-8e79-576870d68ced None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:35.407027 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1241f5-7c96-4c6a-8e79-576870d68ced None admin] PATH_INFO: `/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/646af794697941d6a46d76086e7e8ba9/roles/ff64135ecc4c47ae978c14600bf31853` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.422165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1241f5-7c96-4c6a-8e79-576870d68ced None admin] RBAC: Authorizing `identity:create_grant(project_id=19c2f661cc934aa7869e9b3d4efa875e, user_id=646af794697941d6a46d76086e7e8ba9, role_id=ff64135ecc4c47ae978c14600bf31853)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:35.423622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1241f5-7c96-4c6a-8e79-576870d68ced None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:35.436497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 399/833] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:42:35 2022] PUT /identity/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/646af794697941d6a46d76086e7e8ba9/roles/ff64135ecc4c47ae978c14600bf31853 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:35.437346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1241f5-7c96-4c6a-8e79-576870d68ced None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:35.467902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c7a40-7f85-4e3f-88f7-c4408ea26e3f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:35.509860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de99a1a-a5ab-47bf-9306-2c9669b2af43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:35.510591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de99a1a-a5ab-47bf-9306-2c9669b2af43 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:35.510891 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de99a1a-a5ab-47bf-9306-2c9669b2af43 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:35.511177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de99a1a-a5ab-47bf-9306-2c9669b2af43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.523574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de99a1a-a5ab-47bf-9306-2c9669b2af43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-6de99a1a-a5ab-47bf-9306-2c9669b2af43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:35.531752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:35.533333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 435/834] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:42:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:35.534210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de99a1a-a5ab-47bf-9306-2c9669b2af43 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:35.557734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:35.560246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1241f5-7c96-4c6a-8e79-576870d68ced None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:35.592563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:35.603872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9696c-b0eb-41fa-9352-e146719faf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:35.604547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9696c-b0eb-41fa-9352-e146719faf07 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:35.605148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9696c-b0eb-41fa-9352-e146719faf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:35.605500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9696c-b0eb-41fa-9352-e146719faf07 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.606293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9696c-b0eb-41fa-9352-e146719faf07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:35.606675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9696c-b0eb-41fa-9352-e146719faf07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:35.608602 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9696c-b0eb-41fa-9352-e146719faf07 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:35.622392 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 400/835] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:35.623373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9696c-b0eb-41fa-9352-e146719faf07 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:35.662109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de99a1a-a5ab-47bf-9306-2c9669b2af43 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:35.702711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ad132-44a1-4ca1-8c90-1dd40f5e4669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:35.703377 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ad132-44a1-4ca1-8c90-1dd40f5e4669 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:35.703665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ad132-44a1-4ca1-8c90-1dd40f5e4669 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:35.703944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ad132-44a1-4ca1-8c90-1dd40f5e4669 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.704704 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ad132-44a1-4ca1-8c90-1dd40f5e4669 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:35.704993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ad132-44a1-4ca1-8c90-1dd40f5e4669 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:35.706362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ad132-44a1-4ca1-8c90-1dd40f5e4669 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:35.715254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 436/836] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:42:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:35.715254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ad132-44a1-4ca1-8c90-1dd40f5e4669 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:35.742952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9696c-b0eb-41fa-9352-e146719faf07 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:35.778397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fdc6f-f0fa-4ffb-b7d5-6d23a6712784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:35.779097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fdc6f-f0fa-4ffb-b7d5-6d23a6712784 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:35.779446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fdc6f-f0fa-4ffb-b7d5-6d23a6712784 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:35.779815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fdc6f-f0fa-4ffb-b7d5-6d23a6712784 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.780633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fdc6f-f0fa-4ffb-b7d5-6d23a6712784 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:35.781028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fdc6f-f0fa-4ffb-b7d5-6d23a6712784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:35.782451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fdc6f-f0fa-4ffb-b7d5-6d23a6712784 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:35.791244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 401/837] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:35.792208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fdc6f-f0fa-4ffb-b7d5-6d23a6712784 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:35.807297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:35.831102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ad132-44a1-4ca1-8c90-1dd40f5e4669 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:35.843491 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:35.872989 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ffae8-ae39-4c74-ba26-5bd674a9b305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:35.874523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ffae8-ae39-4c74-ba26-5bd674a9b305 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:35.875019 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ffae8-ae39-4c74-ba26-5bd674a9b305 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:35.875477 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ffae8-ae39-4c74-ba26-5bd674a9b305 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.883854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ffae8-ae39-4c74-ba26-5bd674a9b305 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:35.885397 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ffae8-ae39-4c74-ba26-5bd674a9b305 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:35.899567 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fdc6f-f0fa-4ffb-b7d5-6d23a6712784 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3e1ffae8-ae39-4c74-ba26-5bd674a9b305 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:35.900236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:35.901141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:35.902041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:35.902041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:35.902041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:35.902041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:35.902041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:35.902041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:35.902041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:35.902041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:35.902041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 437/838] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:35.903052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ffae8-ae39-4c74-ba26-5bd674a9b305 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:35.940381 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195567-69f2-4105-b911-619befc4150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:35.941245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195567-69f2-4105-b911-619befc4150f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:35.941358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195567-69f2-4105-b911-619befc4150f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:35.941605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8195567-69f2-4105-b911-619befc4150f None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:35.942307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195567-69f2-4105-b911-619befc4150f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:35.942617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195567-69f2-4105-b911-619befc4150f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:35.943975 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8195567-69f2-4105-b911-619befc4150f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:35.954885 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 402/839] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:42:35 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:35.954885 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195567-69f2-4105-b911-619befc4150f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:36.040070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ffae8-ae39-4c74-ba26-5bd674a9b305 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:36.082242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12d9c0-3fe8-4673-b033-2e31094909eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:36.083166 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12d9c0-3fe8-4673-b033-2e31094909eb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:36.083525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12d9c0-3fe8-4673-b033-2e31094909eb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:36.083881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12d9c0-3fe8-4673-b033-2e31094909eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:36.085185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8195567-69f2-4105-b911-619befc4150f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:36.092101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:36.098012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12d9c0-3fe8-4673-b033-2e31094909eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:36.099490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12d9c0-3fe8-4673-b033-2e31094909eb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:36.100779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3f12d9c0-3fe8-4673-b033-2e31094909eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:36.122926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:36.124427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:36.125357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:36.125357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:36.125357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:36.125357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:36.125357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 438/840] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:36.130158 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9bd10-1e52-4b25-b6f3-dc89afe12048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:36.130447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12d9c0-3fe8-4673-b033-2e31094909eb None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:36.130607 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9bd10-1e52-4b25-b6f3-dc89afe12048 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:36.130607 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9bd10-1e52-4b25-b6f3-dc89afe12048 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:36.130885 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9bd10-1e52-4b25-b6f3-dc89afe12048 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:36.131642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9bd10-1e52-4b25-b6f3-dc89afe12048 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:36.131955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9bd10-1e52-4b25-b6f3-dc89afe12048 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:36.133447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9bd10-1e52-4b25-b6f3-dc89afe12048 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:36.148275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 403/841] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:36.154291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9bd10-1e52-4b25-b6f3-dc89afe12048 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:36.258068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12d9c0-3fe8-4673-b033-2e31094909eb None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:36.270048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9bd10-1e52-4b25-b6f3-dc89afe12048 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:36.298731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd231211-98dd-472b-bee5-1efe5b32239f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:36.299356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd231211-98dd-472b-bee5-1efe5b32239f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:36.299646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd231211-98dd-472b-bee5-1efe5b32239f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:36.299931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd231211-98dd-472b-bee5-1efe5b32239f None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:36.300664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd231211-98dd-472b-bee5-1efe5b32239f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:36.301018 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd231211-98dd-472b-bee5-1efe5b32239f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:36.302399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd231211-98dd-472b-bee5-1efe5b32239f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:36.315449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 439/842] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:36.316245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872104e-e666-41b3-bc56-9a5f7dd675e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:36.317014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872104e-e666-41b3-bc56-9a5f7dd675e3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:36.317302 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872104e-e666-41b3-bc56-9a5f7dd675e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:36.317585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872104e-e666-41b3-bc56-9a5f7dd675e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:36.318421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872104e-e666-41b3-bc56-9a5f7dd675e3 None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=36c18bbd22f74efbae7aab1cdd3f6403) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:36.318737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872104e-e666-41b3-bc56-9a5f7dd675e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:36.320999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872104e-e666-41b3-bc56-9a5f7dd675e3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:36.321188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd231211-98dd-472b-bee5-1efe5b32239f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:36.329314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 404/843] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:36 2022] GET /identity/v3/role_assignments?user.id=36c18bbd22f74efbae7aab1cdd3f6403&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=569707378794437b93d168fc8bb9cbc3 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:36.387584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:36.448176 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd231211-98dd-472b-bee5-1efe5b32239f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:36.486978 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7697d3-b5d5-40a5-a46c-5e0e068ceca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:36.487330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7697d3-b5d5-40a5-a46c-5e0e068ceca7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:36.487540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7697d3-b5d5-40a5-a46c-5e0e068ceca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:36.487822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7697d3-b5d5-40a5-a46c-5e0e068ceca7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:36.496332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7697d3-b5d5-40a5-a46c-5e0e068ceca7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:36.497846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7697d3-b5d5-40a5-a46c-5e0e068ceca7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-fe7697d3-b5d5-40a5-a46c-5e0e068ceca7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:36.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:36.516164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:36.517466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:36.517466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:36.517466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:36.517466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:36.517466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:36.517466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:36.517466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:36.517466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:36.517466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 11:42:36.517466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:36.517466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 440/844] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:42:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:36.521219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872104e-e666-41b3-bc56-9a5f7dd675e3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:36.569007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:36.749373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872104e-e666-41b3-bc56-9a5f7dd675e3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:36.784444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895c7ff-003e-449e-ad20-1c9810ba82ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:36.785136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895c7ff-003e-449e-ad20-1c9810ba82ab None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:36.785417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895c7ff-003e-449e-ad20-1c9810ba82ab None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:36.785691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e895c7ff-003e-449e-ad20-1c9810ba82ab None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:36.786516 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895c7ff-003e-449e-ad20-1c9810ba82ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:36.786822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895c7ff-003e-449e-ad20-1c9810ba82ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:36.788381 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e895c7ff-003e-449e-ad20-1c9810ba82ab None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:36.805953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 405/845] 10.222.0.67 () {56 vars in 1135 bytes} [Thu Jun 23 11:42:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:36.812150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7697d3-b5d5-40a5-a46c-5e0e068ceca7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:36.920481 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7697d3-b5d5-40a5-a46c-5e0e068ceca7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:36.941209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:36.956446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d798e0-75ee-40ea-9e02-1b583bbcbb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:36.957138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d798e0-75ee-40ea-9e02-1b583bbcbb80 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:36.957691 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d798e0-75ee-40ea-9e02-1b583bbcbb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:36.957974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d798e0-75ee-40ea-9e02-1b583bbcbb80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:37.027770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d798e0-75ee-40ea-9e02-1b583bbcbb80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:37.029255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d798e0-75ee-40ea-9e02-1b583bbcbb80 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:37.132341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 441/846] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:42:37.138315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895c7ff-003e-449e-ad20-1c9810ba82ab None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:37.258134 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:37.265012 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e895c7ff-003e-449e-ad20-1c9810ba82ab None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:37.300909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a0a36-4382-42da-9a4b-a8a241e8fac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:37.301749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a0a36-4382-42da-9a4b-a8a241e8fac7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:37.302039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a0a36-4382-42da-9a4b-a8a241e8fac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:37.302318 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a0a36-4382-42da-9a4b-a8a241e8fac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:37.313154 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a0a36-4382-42da-9a4b-a8a241e8fac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:37.314584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a0a36-4382-42da-9a4b-a8a241e8fac7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-123a0a36-4382-42da-9a4b-a8a241e8fac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:37.332185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:37.333113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:37.334018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:37.334018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:37.334018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:37.334018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:37.334018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 406/847] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:37.340509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d798e0-75ee-40ea-9e02-1b583bbcbb80 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:37.353255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:37.452736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d798e0-75ee-40ea-9e02-1b583bbcbb80 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:37.488047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d2bce-a3ba-420f-b38c-b11d37021587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:37.489166 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d2bce-a3ba-420f-b38c-b11d37021587 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:37.489651 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d2bce-a3ba-420f-b38c-b11d37021587 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:37.490125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d2bce-a3ba-420f-b38c-b11d37021587 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:37.491177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d2bce-a3ba-420f-b38c-b11d37021587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:37.491691 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d2bce-a3ba-420f-b38c-b11d37021587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:37.493280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3d2bce-a3ba-420f-b38c-b11d37021587 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:37.506123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 442/848] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:37.510880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99156163-f0f5-4c7d-a1fe-7bef932e89dd None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:37.511119 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99156163-f0f5-4c7d-a1fe-7bef932e89dd None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:37.511402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99156163-f0f5-4c7d-a1fe-7bef932e89dd None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:37.512323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d2bce-a3ba-420f-b38c-b11d37021587 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:37.513015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 407/849] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:37.519597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab382-c158-4859-b566-bfc70dbcd766 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:37.519863 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab382-c158-4859-b566-bfc70dbcd766 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:37.520144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab382-c158-4859-b566-bfc70dbcd766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:37.540300 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-406ab382-c158-4859-b566-bfc70dbcd766 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:37.541553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-406ab382-c158-4859-b566-bfc70dbcd766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:37.594362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:37.616766 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 408/850] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:42:37.635665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3d2bce-a3ba-420f-b38c-b11d37021587 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:37.638064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab382-c158-4859-b566-bfc70dbcd766 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:37.684355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f3564-1b5d-4d88-86d1-a398a4fe07c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:37.685443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f3564-1b5d-4d88-86d1-a398a4fe07c5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:37.686440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f3564-1b5d-4d88-86d1-a398a4fe07c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:37.686959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f3564-1b5d-4d88-86d1-a398a4fe07c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:37.694743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f3564-1b5d-4d88-86d1-a398a4fe07c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:37.696398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f3564-1b5d-4d88-86d1-a398a4fe07c5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c58f3564-1b5d-4d88-86d1-a398a4fe07c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:37.712464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:37.713588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:37.714602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:37.714602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:37.714602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:37.714602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:37.714602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:37.714602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:37.714602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:37.714602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:37.716792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 443/851] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:37.723462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:37.729680 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f3564-1b5d-4d88-86d1-a398a4fe07c5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:37.746457 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab382-c158-4859-b566-bfc70dbcd766 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:37.798563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb7680-d5b3-4f57-8ea3-b2a73ee18660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:37.799277 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb7680-d5b3-4f57-8ea3-b2a73ee18660 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:37.799559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb7680-d5b3-4f57-8ea3-b2a73ee18660 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:37.799835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb7680-d5b3-4f57-8ea3-b2a73ee18660 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:37.814556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceb7680-d5b3-4f57-8ea3-b2a73ee18660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-bceb7680-d5b3-4f57-8ea3-b2a73ee18660 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:37.824448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:37.825546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:37.826417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 409/852] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:42:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:37.831311 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb7680-d5b3-4f57-8ea3-b2a73ee18660 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:37.865839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f3564-1b5d-4d88-86d1-a398a4fe07c5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:37.904586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:37.912978 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad497b1-73d4-4123-9d52-d5daed155a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:37.914110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad497b1-73d4-4123-9d52-d5daed155a91 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:37.914574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad497b1-73d4-4123-9d52-d5daed155a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:37.915001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad497b1-73d4-4123-9d52-d5daed155a91 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:37.916422 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad497b1-73d4-4123-9d52-d5daed155a91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:37.916908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad497b1-73d4-4123-9d52-d5daed155a91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:37.918917 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad497b1-73d4-4123-9d52-d5daed155a91 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:37.940027 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb7680-d5b3-4f57-8ea3-b2a73ee18660 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:37.940654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 444/853] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:37.942619 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad497b1-73d4-4123-9d52-d5daed155a91 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:37.975786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab685a2e-9500-47e2-b21a-aa9bb6204b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:37.976702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab685a2e-9500-47e2-b21a-aa9bb6204b37 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:37.977356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab685a2e-9500-47e2-b21a-aa9bb6204b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:37.977621 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab685a2e-9500-47e2-b21a-aa9bb6204b37 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:37.978529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab685a2e-9500-47e2-b21a-aa9bb6204b37 None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:37.978885 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab685a2e-9500-47e2-b21a-aa9bb6204b37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:37.980868 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab685a2e-9500-47e2-b21a-aa9bb6204b37 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:37.993049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:37.994928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 410/854] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:42:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:37.996047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab685a2e-9500-47e2-b21a-aa9bb6204b37 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:38.076496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad497b1-73d4-4123-9d52-d5daed155a91 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:38.116839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b2a39f-62ab-47ad-80a0-71eca047856f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:38.117491 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2a39f-62ab-47ad-80a0-71eca047856f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:38.117774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2a39f-62ab-47ad-80a0-71eca047856f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:38.117979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab685a2e-9500-47e2-b21a-aa9bb6204b37 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:38.118083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2a39f-62ab-47ad-80a0-71eca047856f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:38.118867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2a39f-62ab-47ad-80a0-71eca047856f None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=eb9b9af5e0b44e34837e1cdb5ff6d33e) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:38.119179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2a39f-62ab-47ad-80a0-71eca047856f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:38.120596 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b2a39f-62ab-47ad-80a0-71eca047856f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:38.126632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 445/855] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:37 2022] GET /identity/v3/role_assignments?user.id=eb9b9af5e0b44e34837e1cdb5ff6d33e&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=569707378794437b93d168fc8bb9cbc3 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:38.127892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef7198-cf14-4397-ab41-336c6f3ea520 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:38.128181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef7198-cf14-4397-ab41-336c6f3ea520 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:38.128459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aef7198-cf14-4397-ab41-336c6f3ea520 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:38.132712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 446/856] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:38.135374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38af69f-4564-4d81-9669-1f50517bfd94 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:38.135892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38af69f-4564-4d81-9669-1f50517bfd94 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:38.136349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38af69f-4564-4d81-9669-1f50517bfd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:38.154402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d38af69f-4564-4d81-9669-1f50517bfd94 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:38.156511 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d38af69f-4564-4d81-9669-1f50517bfd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:38.173645 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2daff3-b7fc-487d-8267-5c649cbdbc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:38.175063 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2daff3-b7fc-487d-8267-5c649cbdbc6c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:38.175435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2daff3-b7fc-487d-8267-5c649cbdbc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:38.175798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2daff3-b7fc-487d-8267-5c649cbdbc6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:38.191678 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2daff3-b7fc-487d-8267-5c649cbdbc6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:38.193614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2daff3-b7fc-487d-8267-5c649cbdbc6c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-4d2daff3-b7fc-487d-8267-5c649cbdbc6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:38.213156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:38.214133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:38.215031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:38.215031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:38.215031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:38.215031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:38.218126 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 411/857] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:38.222592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:38.224579 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2daff3-b7fc-487d-8267-5c649cbdbc6c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:38.256851 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 447/858] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:42:38.272102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38af69f-4564-4d81-9669-1f50517bfd94 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:38.290646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:38.345907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2daff3-b7fc-487d-8267-5c649cbdbc6c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:38.389366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fc926-55cc-4672-9507-8ea77d4d1bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:38.390035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fc926-55cc-4672-9507-8ea77d4d1bfb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:38.390324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fc926-55cc-4672-9507-8ea77d4d1bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:38.390606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fc926-55cc-4672-9507-8ea77d4d1bfb None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:38.391336 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fc926-55cc-4672-9507-8ea77d4d1bfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:38.391648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fc926-55cc-4672-9507-8ea77d4d1bfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:38.393059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fc926-55cc-4672-9507-8ea77d4d1bfb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:38.399024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38af69f-4564-4d81-9669-1f50517bfd94 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:38.407550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 412/859] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:38.415102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fc926-55cc-4672-9507-8ea77d4d1bfb None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:38.440371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdbf5a-e743-4350-b536-3d762848dd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:38.441080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdbf5a-e743-4350-b536-3d762848dd00 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:38.441368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdbf5a-e743-4350-b536-3d762848dd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:38.441647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdbf5a-e743-4350-b536-3d762848dd00 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:38.442443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdbf5a-e743-4350-b536-3d762848dd00 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:38.444005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdbf5a-e743-4350-b536-3d762848dd00 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:38.461187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 448/860] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:42:38 2022] POST /identity/v3/services => generated 251 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:42:38.539163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fc926-55cc-4672-9507-8ea77d4d1bfb None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:38.559025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:38.590309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033496f5-a065-4a32-807b-677e016c4185 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:38.590625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033496f5-a065-4a32-807b-677e016c4185 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:38.590889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033496f5-a065-4a32-807b-677e016c4185 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:38.590993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d418d-e77a-40bb-bc67-76a383191cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:38.591747 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d418d-e77a-40bb-bc67-76a383191cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:38.592078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d418d-e77a-40bb-bc67-76a383191cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:38.592355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d418d-e77a-40bb-bc67-76a383191cc6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:38.592616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 449/861] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:42:38.598209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88795194-9a43-4517-aa61-3fe0dd74614d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:38.598490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88795194-9a43-4517-aa61-3fe0dd74614d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:38.598690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88795194-9a43-4517-aa61-3fe0dd74614d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:38.601253 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d418d-e77a-40bb-bc67-76a383191cc6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:38.602825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d418d-e77a-40bb-bc67-76a383191cc6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:38.618977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-88795194-9a43-4517-aa61-3fe0dd74614d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:38.620197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-88795194-9a43-4517-aa61-3fe0dd74614d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-b52d418d-e77a-40bb-bc67-76a383191cc6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:38.625075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:38.626038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:38.627002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:38.627002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:38.627002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:38.627002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:38.627002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:38.627002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:38.627002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:38.627002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:38.627002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:42:38.627002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:38.627002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 413/862] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 23 11:42:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:38.634094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d418d-e77a-40bb-bc67-76a383191cc6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:38.689803 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:38.729922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 450/863] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:38 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:42:38.759947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88795194-9a43-4517-aa61-3fe0dd74614d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:38.763140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d418d-e77a-40bb-bc67-76a383191cc6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:38.799952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae394bf-fb8d-45dd-a32a-bece31508a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:38.801055 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae394bf-fb8d-45dd-a32a-bece31508a96 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:38.801578 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae394bf-fb8d-45dd-a32a-bece31508a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:38.802078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae394bf-fb8d-45dd-a32a-bece31508a96 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:38.803135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae394bf-fb8d-45dd-a32a-bece31508a96 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:38.803652 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae394bf-fb8d-45dd-a32a-bece31508a96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:38.805316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae394bf-fb8d-45dd-a32a-bece31508a96 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:38.825622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 414/864] 10.222.0.67 () {56 vars in 1149 bytes} [Thu Jun 23 11:42:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:38.832725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae394bf-fb8d-45dd-a32a-bece31508a96 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:38.870346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:38.873991 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88795194-9a43-4517-aa61-3fe0dd74614d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:38.914690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83c5a5-2e59-408f-8583-f7a460777734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:38.916141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83c5a5-2e59-408f-8583-f7a460777734 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:38.916603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83c5a5-2e59-408f-8583-f7a460777734 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:38.917093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83c5a5-2e59-408f-8583-f7a460777734 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:38.918097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d83c5a5-2e59-408f-8583-f7a460777734 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:38.920994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d83c5a5-2e59-408f-8583-f7a460777734 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3d83c5a5-2e59-408f-8583-f7a460777734 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:38.937140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:38.938107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:42:38.938945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:38.941815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 451/865] 10.222.0.67 () {56 vars in 1132 bytes} [Thu Jun 23 11:42:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:38.947443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83c5a5-2e59-408f-8583-f7a460777734 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:38.963187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae394bf-fb8d-45dd-a32a-bece31508a96 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:38.990238 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:39.009601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d61bb0-8501-435c-b5a4-6bfb05105c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:39.010636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d61bb0-8501-435c-b5a4-6bfb05105c1b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:39.011132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d61bb0-8501-435c-b5a4-6bfb05105c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:39.011638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d61bb0-8501-435c-b5a4-6bfb05105c1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:39.061381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83c5a5-2e59-408f-8583-f7a460777734 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:39.083204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d61bb0-8501-435c-b5a4-6bfb05105c1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:39.085100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d61bb0-8501-435c-b5a4-6bfb05105c1b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:39.098240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834f39-7c19-4749-b59e-0a30037ed6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:39.098956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834f39-7c19-4749-b59e-0a30037ed6be None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:39.099199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834f39-7c19-4749-b59e-0a30037ed6be None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:39.099485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b834f39-7c19-4749-b59e-0a30037ed6be None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:39.100195 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b834f39-7c19-4749-b59e-0a30037ed6be None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:39.100510 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b834f39-7c19-4749-b59e-0a30037ed6be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:39.102025 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b834f39-7c19-4749-b59e-0a30037ed6be None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:39.112600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 452/866] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 23 11:42:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:39.113916 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834f39-7c19-4749-b59e-0a30037ed6be None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:39.115243 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:39.195092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 415/867] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:38 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:42:39.196227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d61bb0-8501-435c-b5a4-6bfb05105c1b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:39.226618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b834f39-7c19-4749-b59e-0a30037ed6be None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:39.228490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:39.277300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1478c0-1da9-41d5-978e-57140ac841be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:39.278479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1478c0-1da9-41d5-978e-57140ac841be None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:39.278916 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1478c0-1da9-41d5-978e-57140ac841be None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:39.279375 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1478c0-1da9-41d5-978e-57140ac841be None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:39.280404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1478c0-1da9-41d5-978e-57140ac841be None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:39.281263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1478c0-1da9-41d5-978e-57140ac841be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:39.283344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1478c0-1da9-41d5-978e-57140ac841be None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:39.291068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 453/868] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 23 11:42:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:39.303008 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d61bb0-8501-435c-b5a4-6bfb05105c1b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:39.335550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:39.346534 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992044b7-32ce-4597-9cb5-795ca785416a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:39.347757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992044b7-32ce-4597-9cb5-795ca785416a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:39.348246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992044b7-32ce-4597-9cb5-795ca785416a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:39.348799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992044b7-32ce-4597-9cb5-795ca785416a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:39.362938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992044b7-32ce-4597-9cb5-795ca785416a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:39.364876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992044b7-32ce-4597-9cb5-795ca785416a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-992044b7-32ce-4597-9cb5-795ca785416a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:39.385910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:39.387073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:39.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:39.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:39.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:39.388179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:39.390291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 416/869] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:39.396260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1478c0-1da9-41d5-978e-57140ac841be None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:39.512085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1478c0-1da9-41d5-978e-57140ac841be None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:39.550072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99c6c5-d1e7-4183-a030-26902fb1546f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:39.550741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99c6c5-d1e7-4183-a030-26902fb1546f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:39.551024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99c6c5-d1e7-4183-a030-26902fb1546f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:39.551302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f99c6c5-d1e7-4183-a030-26902fb1546f None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:39.552061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99c6c5-d1e7-4183-a030-26902fb1546f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:39.552368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99c6c5-d1e7-4183-a030-26902fb1546f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:39.553773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f99c6c5-d1e7-4183-a030-26902fb1546f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:39.567041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 454/870] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:39.572362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992044b7-32ce-4597-9cb5-795ca785416a None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:39.675586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992044b7-32ce-4597-9cb5-795ca785416a None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:39.710230 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eea7f9-c417-4931-a603-683f5f568372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:39.711030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eea7f9-c417-4931-a603-683f5f568372 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:39.711316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eea7f9-c417-4931-a603-683f5f568372 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:39.711592 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eea7f9-c417-4931-a603-683f5f568372 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:39.719399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eea7f9-c417-4931-a603-683f5f568372 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:39.720880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eea7f9-c417-4931-a603-683f5f568372 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-20eea7f9-c417-4931-a603-683f5f568372 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:39.735673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:39.736618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:39.737539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:39.737539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:39.737539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:39.737539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:39.737539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:39.737539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:39.737539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:39.737539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:39.737539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 417/871] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:39.743146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99c6c5-d1e7-4183-a030-26902fb1546f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:39.772643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:39.852114 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f99c6c5-d1e7-4183-a030-26902fb1546f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:39.887866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0455f9-e467-4d6a-a697-5550497a2dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:39.888631 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0455f9-e467-4d6a-a697-5550497a2dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:39.888866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0455f9-e467-4d6a-a697-5550497a2dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:39.889143 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0455f9-e467-4d6a-a697-5550497a2dd0 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:39.889938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0455f9-e467-4d6a-a697-5550497a2dd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:39.890254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0455f9-e467-4d6a-a697-5550497a2dd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:39.891687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0455f9-e467-4d6a-a697-5550497a2dd0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:39.904127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 455/872] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:39.910416 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eea7f9-c417-4931-a603-683f5f568372 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:39.952875 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:40.015094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eea7f9-c417-4931-a603-683f5f568372 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:40.050347 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff12d10a-db48-48d2-adbe-c25b801dd420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:40.050929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12d10a-db48-48d2-adbe-c25b801dd420 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:40.051213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12d10a-db48-48d2-adbe-c25b801dd420 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:40.051490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff12d10a-db48-48d2-adbe-c25b801dd420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:40.052328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12d10a-db48-48d2-adbe-c25b801dd420 None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=646af794697941d6a46d76086e7e8ba9) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:40.052647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12d10a-db48-48d2-adbe-c25b801dd420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:40.054123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff12d10a-db48-48d2-adbe-c25b801dd420 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:40.065018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 418/873] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:39 2022] GET /identity/v3/role_assignments?user.id=646af794697941d6a46d76086e7e8ba9&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:40.358338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2c955-3976-4257-b612-e6cdcfcc5c4d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:40.359000 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2c955-3976-4257-b612-e6cdcfcc5c4d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:40.359505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d2c955-3976-4257-b612-e6cdcfcc5c4d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:40.361570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 456/874] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:40.367694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22baf5-fac6-437a-bed0-b453e57a37bb None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:40.368252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22baf5-fac6-437a-bed0-b453e57a37bb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:40.368788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22baf5-fac6-437a-bed0-b453e57a37bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:40.386907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0c22baf5-fac6-437a-bed0-b453e57a37bb None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:40.388417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0c22baf5-fac6-437a-bed0-b453e57a37bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:40.457216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 419/875] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:40 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:40.480010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2c955-3976-4257-b612-e6cdcfcc5c4d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:40.577171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:40.597441 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d2c955-3976-4257-b612-e6cdcfcc5c4d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:40.633884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0502c31-2d12-4e5d-9ef8-95583a8a39d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:40.634607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0502c31-2d12-4e5d-9ef8-95583a8a39d0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:40.634886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0502c31-2d12-4e5d-9ef8-95583a8a39d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:40.635159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0502c31-2d12-4e5d-9ef8-95583a8a39d0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:40.635818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0502c31-2d12-4e5d-9ef8-95583a8a39d0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:40.637208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0502c31-2d12-4e5d-9ef8-95583a8a39d0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-d0502c31-2d12-4e5d-9ef8-95583a8a39d0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:40.650094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:40.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:40.652029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 457/876] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 23 11:42:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:40.657455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22baf5-fac6-437a-bed0-b453e57a37bb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:40.693350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:40.702634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ad092-85c8-4913-b723-229ee9c03891 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:40.703567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ad092-85c8-4913-b723-229ee9c03891 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:40.704409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ad092-85c8-4913-b723-229ee9c03891 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:40.707486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 458/877] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:40.712592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a785e-6d58-4b30-bf29-2737d41d93be None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:40.713110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a785e-6d58-4b30-bf29-2737d41d93be None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:40.713399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a785e-6d58-4b30-bf29-2737d41d93be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:40.731953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-527a785e-6d58-4b30-bf29-2737d41d93be None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:40.734702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-527a785e-6d58-4b30-bf29-2737d41d93be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:40.775957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22baf5-fac6-437a-bed0-b453e57a37bb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:40.813727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0930d4a-0e99-4dcf-9c3a-6c1210e3d9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:40.814637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0930d4a-0e99-4dcf-9c3a-6c1210e3d9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:40.815161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0930d4a-0e99-4dcf-9c3a-6c1210e3d9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:40.815639 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0930d4a-0e99-4dcf-9c3a-6c1210e3d9e1 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:40.816543 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0930d4a-0e99-4dcf-9c3a-6c1210e3d9e1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:40.817085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0930d4a-0e99-4dcf-9c3a-6c1210e3d9e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:40.818651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0930d4a-0e99-4dcf-9c3a-6c1210e3d9e1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:40.824301 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 420/878] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 23 11:42:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:40.830691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0930d4a-0e99-4dcf-9c3a-6c1210e3d9e1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:40.842509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 459/879] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:40 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:40.862671 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a785e-6d58-4b30-bf29-2737d41d93be None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:40.948860 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:40.955007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:40.957767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0930d4a-0e99-4dcf-9c3a-6c1210e3d9e1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:40.976988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a785e-6d58-4b30-bf29-2737d41d93be None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:41.001477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c759c44-7958-4080-8324-c5bfda6abf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:41.002141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c759c44-7958-4080-8324-c5bfda6abf98 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:41.002602 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c759c44-7958-4080-8324-c5bfda6abf98 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:41.003201 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c759c44-7958-4080-8324-c5bfda6abf98 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:41.004388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c759c44-7958-4080-8324-c5bfda6abf98 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:41.005032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c759c44-7958-4080-8324-c5bfda6abf98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:41.007562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c759c44-7958-4080-8324-c5bfda6abf98 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:41.024898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 421/880] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 23 11:42:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:41.028391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db344dd-275b-4207-a4e0-1a9b297a7dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:41.029260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db344dd-275b-4207-a4e0-1a9b297a7dda None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:41.029642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db344dd-275b-4207-a4e0-1a9b297a7dda None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:41.029921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db344dd-275b-4207-a4e0-1a9b297a7dda None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:41.030644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db344dd-275b-4207-a4e0-1a9b297a7dda None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:41.032048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db344dd-275b-4207-a4e0-1a9b297a7dda None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3db344dd-275b-4207-a4e0-1a9b297a7dda None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:41.046142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:41.047262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:41.048117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 460/881] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:42:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:41.053980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c759c44-7958-4080-8324-c5bfda6abf98 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:41.164897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c759c44-7958-4080-8324-c5bfda6abf98 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:41.203995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3151c37-4df0-4796-b23d-1ce6d9119c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:41.203995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3151c37-4df0-4796-b23d-1ce6d9119c65 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:41.203995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3151c37-4df0-4796-b23d-1ce6d9119c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:41.204326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3151c37-4df0-4796-b23d-1ce6d9119c65 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:41.204778 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3151c37-4df0-4796-b23d-1ce6d9119c65 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:41.205127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3151c37-4df0-4796-b23d-1ce6d9119c65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:41.206613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3151c37-4df0-4796-b23d-1ce6d9119c65 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:41.211879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 422/882] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:42:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:41.219793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db344dd-275b-4207-a4e0-1a9b297a7dda None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:41.333917 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db344dd-275b-4207-a4e0-1a9b297a7dda None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:41.372661 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f16a3-7b88-4c38-a636-6469224b2848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:41.374246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f16a3-7b88-4c38-a636-6469224b2848 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:41.374768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f16a3-7b88-4c38-a636-6469224b2848 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:41.375256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f16a3-7b88-4c38-a636-6469224b2848 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:41.376180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f16a3-7b88-4c38-a636-6469224b2848 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:41.376714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f16a3-7b88-4c38-a636-6469224b2848 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:41.378284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f16a3-7b88-4c38-a636-6469224b2848 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:41.384094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 461/883] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:42:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:41.390205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3151c37-4df0-4796-b23d-1ce6d9119c65 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:41.418718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:41.499304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3151c37-4df0-4796-b23d-1ce6d9119c65 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:41.503903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf5e38-5bc6-40ad-b8a9-a849b3e9fe6a None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:41.504198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf5e38-5bc6-40ad-b8a9-a849b3e9fe6a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:41.504486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf5e38-5bc6-40ad-b8a9-a849b3e9fe6a None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:41.506126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 462/884] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:42:41.511225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81998bac-8687-4f7f-a250-a9107b7b5b6e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:41.511518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81998bac-8687-4f7f-a250-a9107b7b5b6e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:41.511805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81998bac-8687-4f7f-a250-a9107b7b5b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:41.530034 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-81998bac-8687-4f7f-a250-a9107b7b5b6e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:41.531254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-81998bac-8687-4f7f-a250-a9107b7b5b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:41.533776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:41.538806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15771d61-b791-47b9-b77a-79cdaccc83ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:41.539736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15771d61-b791-47b9-b77a-79cdaccc83ca None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:41.540243 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15771d61-b791-47b9-b77a-79cdaccc83ca None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:41.540746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15771d61-b791-47b9-b77a-79cdaccc83ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:41.541730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15771d61-b791-47b9-b77a-79cdaccc83ca None admin] RBAC: Adding query filter params (interface=public, service_id=1e1e1414dc81420c8336f9e7434902c6, region_id=RegionOne) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:41.542257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15771d61-b791-47b9-b77a-79cdaccc83ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:41.543831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15771d61-b791-47b9-b77a-79cdaccc83ca None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:41.550254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 423/885] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 23 11:42:41 2022] GET /identity/v3/endpoints?service_id=1e1e1414dc81420c8336f9e7434902c6&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:41.558727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15771d61-b791-47b9-b77a-79cdaccc83ca None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:41.620626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 463/886] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:41.643648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81998bac-8687-4f7f-a250-a9107b7b5b6e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:41.670799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:41.686829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15771d61-b791-47b9-b77a-79cdaccc83ca None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:41.726362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d680a-5c7c-4e97-90e6-fd0a25ff07fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:41.727928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d680a-5c7c-4e97-90e6-fd0a25ff07fe None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:41.728835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d680a-5c7c-4e97-90e6-fd0a25ff07fe None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:41.729504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d680a-5c7c-4e97-90e6-fd0a25ff07fe None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:41.730473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d680a-5c7c-4e97-90e6-fd0a25ff07fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:41.733185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d680a-5c7c-4e97-90e6-fd0a25ff07fe None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:41.744958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 424/887] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 23 11:42:41 2022] GET /identity/v3/services => generated 1011 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:42:41.749188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81998bac-8687-4f7f-a250-a9107b7b5b6e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:41.788390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efae54-1a69-4e58-a2ae-f6b773a13bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:41.789014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efae54-1a69-4e58-a2ae-f6b773a13bdf None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:41.789306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efae54-1a69-4e58-a2ae-f6b773a13bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:41.789562 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efae54-1a69-4e58-a2ae-f6b773a13bdf None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:41.790309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efae54-1a69-4e58-a2ae-f6b773a13bdf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:41.791701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efae54-1a69-4e58-a2ae-f6b773a13bdf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:41.803072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:41.810662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 464/888] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:42:41 2022] POST /identity/v3/services => generated 269 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:42:42.239929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690d51c-53fc-423e-b0e3-e07badd74b43 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:42.241943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690d51c-53fc-423e-b0e3-e07badd74b43 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:42.241943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690d51c-53fc-423e-b0e3-e07badd74b43 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:42.242408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 425/889] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:42.247923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939a59c-2b5a-4c8e-9076-8f6aabf754e7 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:42.249423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939a59c-2b5a-4c8e-9076-8f6aabf754e7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:42.249907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0939a59c-2b5a-4c8e-9076-8f6aabf754e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:42.267545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0939a59c-2b5a-4c8e-9076-8f6aabf754e7 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:42.269191 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0939a59c-2b5a-4c8e-9076-8f6aabf754e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:42.346539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 465/890] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:42.370948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690d51c-53fc-423e-b0e3-e07badd74b43 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:42.478851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690d51c-53fc-423e-b0e3-e07badd74b43 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:42.514870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51de3db-81b1-46c2-a497-dc5ad8a26a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:42.516011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51de3db-81b1-46c2-a497-dc5ad8a26a7b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:42.516582 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51de3db-81b1-46c2-a497-dc5ad8a26a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:42.517089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51de3db-81b1-46c2-a497-dc5ad8a26a7b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:42.528932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51de3db-81b1-46c2-a497-dc5ad8a26a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-a51de3db-81b1-46c2-a497-dc5ad8a26a7b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:42.538320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:42.539244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:42.540082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:42.540082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:42.540082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:42.540082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:42.540082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:42.540082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:42.540082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:42.540082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:42.540082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:42.540082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:42.540082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:42:42.540082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:42.542876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 426/891] 10.222.0.67 () {56 vars in 1120 bytes} [Thu Jun 23 11:42:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:42.549031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939a59c-2b5a-4c8e-9076-8f6aabf754e7 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:42.655776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0939a59c-2b5a-4c8e-9076-8f6aabf754e7 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:42.689084 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:42.697778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40c4fc-8842-4ef0-9cbd-9e64e68e5cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:42.698772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40c4fc-8842-4ef0-9cbd-9e64e68e5cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:42.699392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40c4fc-8842-4ef0-9cbd-9e64e68e5cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:42.700136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c40c4fc-8842-4ef0-9cbd-9e64e68e5cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:42.700955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40c4fc-8842-4ef0-9cbd-9e64e68e5cd5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:42.701323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40c4fc-8842-4ef0-9cbd-9e64e68e5cd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:42.703213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c40c4fc-8842-4ef0-9cbd-9e64e68e5cd5 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:42.711957 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 466/892] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 23 11:42:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:42.717555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51de3db-81b1-46c2-a497-dc5ad8a26a7b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:42.734533 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:42.829273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51de3db-81b1-46c2-a497-dc5ad8a26a7b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:42.863686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd1ef7-f92e-494e-af28-70e0c0807fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:42.864478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd1ef7-f92e-494e-af28-70e0c0807fcc None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:42.864803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd1ef7-f92e-494e-af28-70e0c0807fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:42.865088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd1ef7-f92e-494e-af28-70e0c0807fcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:42.877224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd1ef7-f92e-494e-af28-70e0c0807fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:42.878676 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd1ef7-f92e-494e-af28-70e0c0807fcc None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-1ddd1ef7-f92e-494e-af28-70e0c0807fcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:42.897088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:42.898014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:42.898900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:42.898900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:42.898900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:42.898900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:42.898900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 427/893] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:42.905798 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40c4fc-8842-4ef0-9cbd-9e64e68e5cd5 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:43.020458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c40c4fc-8842-4ef0-9cbd-9e64e68e5cd5 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:43.056272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db305dd-3ebd-46bc-8392-d146f66e1d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:43.056993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db305dd-3ebd-46bc-8392-d146f66e1d2f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:43.057283 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db305dd-3ebd-46bc-8392-d146f66e1d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:43.057568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db305dd-3ebd-46bc-8392-d146f66e1d2f None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:43.058292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db305dd-3ebd-46bc-8392-d146f66e1d2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:43.058604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db305dd-3ebd-46bc-8392-d146f66e1d2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:43.059969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db305dd-3ebd-46bc-8392-d146f66e1d2f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:43.073070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 467/894] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:43.079070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd1ef7-f92e-494e-af28-70e0c0807fcc None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:43.150792 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:43.186194 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd1ef7-f92e-494e-af28-70e0c0807fcc None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:43.223086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0600f-29e5-4f27-8a4f-50281290624b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:43.223947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0600f-29e5-4f27-8a4f-50281290624b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:43.224228 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0600f-29e5-4f27-8a4f-50281290624b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:43.224553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0600f-29e5-4f27-8a4f-50281290624b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:43.233146 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0600f-29e5-4f27-8a4f-50281290624b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:43.234653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0600f-29e5-4f27-8a4f-50281290624b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-beb0600f-29e5-4f27-8a4f-50281290624b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:43.252647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:43.253758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:43.254805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:43.254805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:43.254805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:43.254805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:43.254805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:43.254805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:43.254805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:43.254805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:43.254805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:42:43.254805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:43.254805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 428/895] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 23 11:42:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:43.255447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1ed37-0569-4fc2-a2d8-1ab7f545bca8 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:43.255447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1ed37-0569-4fc2-a2d8-1ab7f545bca8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:43.255447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1ed37-0569-4fc2-a2d8-1ab7f545bca8 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:43.256336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 468/896] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:43.262101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1ed37-0569-4fc2-a2d8-1ab7f545bca8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:43.262239 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c1884-2c0c-43c9-95a1-9a2c21958443 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:43.262538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c1884-2c0c-43c9-95a1-9a2c21958443 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:43.262819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c1884-2c0c-43c9-95a1-9a2c21958443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:43.280196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c09c1884-2c0c-43c9-95a1-9a2c21958443 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:43.281412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c09c1884-2c0c-43c9-95a1-9a2c21958443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:43.323591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:43.363229 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 429/897] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:43.386503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c1884-2c0c-43c9-95a1-9a2c21958443 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:43.390875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1ed37-0569-4fc2-a2d8-1ab7f545bca8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:43.428609 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a3746-ba38-4e9b-91ff-bc337a33d6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:43.429536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a3746-ba38-4e9b-91ff-bc337a33d6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:43.429824 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a3746-ba38-4e9b-91ff-bc337a33d6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:43.430104 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a3746-ba38-4e9b-91ff-bc337a33d6a2 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:43.430952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a3746-ba38-4e9b-91ff-bc337a33d6a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:43.431266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a3746-ba38-4e9b-91ff-bc337a33d6a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:43.432858 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a3746-ba38-4e9b-91ff-bc337a33d6a2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:43.447937 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:43.458837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 469/898] 10.222.0.67 () {56 vars in 1149 bytes} [Thu Jun 23 11:42:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:43.460262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a3746-ba38-4e9b-91ff-bc337a33d6a2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:43.510154 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c1884-2c0c-43c9-95a1-9a2c21958443 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:43.554305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa5c76-b0c9-415f-9767-611fe0bba7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:43.555490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa5c76-b0c9-415f-9767-611fe0bba7ed None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:43.556070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa5c76-b0c9-415f-9767-611fe0bba7ed None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:43.556551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafa5c76-b0c9-415f-9767-611fe0bba7ed None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:43.557512 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa5c76-b0c9-415f-9767-611fe0bba7ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:43.559197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafa5c76-b0c9-415f-9767-611fe0bba7ed None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:43.573509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a3746-ba38-4e9b-91ff-bc337a33d6a2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:43.581265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 430/899] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:42:43 2022] POST /identity/v3/services => generated 241 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:42:43.581878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:43.619768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412020c-f9d7-42d5-adeb-71a9b23628c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:43.620434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412020c-f9d7-42d5-adeb-71a9b23628c3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:43.620746 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412020c-f9d7-42d5-adeb-71a9b23628c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:43.621029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412020c-f9d7-42d5-adeb-71a9b23628c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:43.698487 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412020c-f9d7-42d5-adeb-71a9b23628c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:43.699878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412020c-f9d7-42d5-adeb-71a9b23628c3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:43.807160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:43.811104 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 470/900] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:43 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 11:42:43.817613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa5c76-b0c9-415f-9767-611fe0bba7ed None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:43.927321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafa5c76-b0c9-415f-9767-611fe0bba7ed None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:43.936387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accf1b0-30d9-4778-86d6-99fe0f176edf None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:43.936705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accf1b0-30d9-4778-86d6-99fe0f176edf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:43.936982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accf1b0-30d9-4778-86d6-99fe0f176edf None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:43.939624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 471/901] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:42:43.942916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:43.943739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93763d78-05fc-4b54-9fe1-454e25e93d51 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:43.944035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93763d78-05fc-4b54-9fe1-454e25e93d51 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:43.944424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93763d78-05fc-4b54-9fe1-454e25e93d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:43.964758 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-93763d78-05fc-4b54-9fe1-454e25e93d51 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:43.966357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-93763d78-05fc-4b54-9fe1-454e25e93d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:43.969547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8084141c-f5ed-489f-b6d0-957b6223ea1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:43.970344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084141c-f5ed-489f-b6d0-957b6223ea1e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:43.970628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084141c-f5ed-489f-b6d0-957b6223ea1e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:43.970905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8084141c-f5ed-489f-b6d0-957b6223ea1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:43.981644 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084141c-f5ed-489f-b6d0-957b6223ea1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:43.983689 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8084141c-f5ed-489f-b6d0-957b6223ea1e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-8084141c-f5ed-489f-b6d0-957b6223ea1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:44.003028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:44.004106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:44.005255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:44.005255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:44.005255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:44.005255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:44.007101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 431/902] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:44.008502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26345a-181d-4947-ba1e-2ee4bbd6e1c2 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:44.008964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26345a-181d-4947-ba1e-2ee4bbd6e1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:44.009389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26345a-181d-4947-ba1e-2ee4bbd6e1c2 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:44.011028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 432/903] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:44.012076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26345a-181d-4947-ba1e-2ee4bbd6e1c2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:44.040901 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 472/904] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:42:44.042548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bbe8a-6546-49d1-baa5-0ecb7f6531b7 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:44.042836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bbe8a-6546-49d1-baa5-0ecb7f6531b7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:44.043117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bbe8a-6546-49d1-baa5-0ecb7f6531b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:44.067564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-672bbe8a-6546-49d1-baa5-0ecb7f6531b7 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:44.068961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-672bbe8a-6546-49d1-baa5-0ecb7f6531b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:44.130749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26345a-181d-4947-ba1e-2ee4bbd6e1c2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:44.140445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 473/905] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:44.140853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bbe8a-6546-49d1-baa5-0ecb7f6531b7 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:44.173569 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3e6e6-e6d6-403e-bf2f-e900f9bd29ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:44.174238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3e6e6-e6d6-403e-bf2f-e900f9bd29ee None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:44.174526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3e6e6-e6d6-403e-bf2f-e900f9bd29ee None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:44.174921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3e6e6-e6d6-403e-bf2f-e900f9bd29ee None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:44.175749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3e6e6-e6d6-403e-bf2f-e900f9bd29ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:44.175964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3e6e6-e6d6-403e-bf2f-e900f9bd29ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:44.177365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3e6e6-e6d6-403e-bf2f-e900f9bd29ee None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:44.195530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 433/906] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:44.195530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3e6e6-e6d6-403e-bf2f-e900f9bd29ee None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:44.216744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:44.223681 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:44.264576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bbe8a-6546-49d1-baa5-0ecb7f6531b7 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:44.307057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ecfb9-2b85-496f-94fd-a3f66148dc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:44.307870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ecfb9-2b85-496f-94fd-a3f66148dc60 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:44.308163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ecfb9-2b85-496f-94fd-a3f66148dc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:44.308443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ecfb9-2b85-496f-94fd-a3f66148dc60 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:44.309155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ecfb9-2b85-496f-94fd-a3f66148dc60 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:44.310517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ecfb9-2b85-496f-94fd-a3f66148dc60 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:44.311924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3e6e6-e6d6-403e-bf2f-e900f9bd29ee None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-fd9ecfb9-2b85-496f-94fd-a3f66148dc60 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:44.323846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:44.324907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:44.325801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 474/907] 10.222.0.67 () {56 vars in 1132 bytes} [Thu Jun 23 11:42:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:44.327284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ecfb9-2b85-496f-94fd-a3f66148dc60 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:44.356462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae225ce-b2bd-466c-8532-2c793960483c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:44.357334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae225ce-b2bd-466c-8532-2c793960483c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:44.357770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae225ce-b2bd-466c-8532-2c793960483c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:44.358054 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae225ce-b2bd-466c-8532-2c793960483c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:44.358725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae225ce-b2bd-466c-8532-2c793960483c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:44.360116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae225ce-b2bd-466c-8532-2c793960483c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-3ae225ce-b2bd-466c-8532-2c793960483c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:44.372617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:44.374305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 11:42:44.375148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:44.379393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 434/908] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:42:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:44.380285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae225ce-b2bd-466c-8532-2c793960483c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:44.449786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ecfb9-2b85-496f-94fd-a3f66148dc60 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:44.475647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:44.485795 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:44.488251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5db7b-c840-41eb-9d00-68798c724ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:44.489357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5db7b-c840-41eb-9d00-68798c724ace None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:44.489644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5db7b-c840-41eb-9d00-68798c724ace None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:44.489931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de5db7b-c840-41eb-9d00-68798c724ace None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:44.496414 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae225ce-b2bd-466c-8532-2c793960483c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:44.496788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5db7b-c840-41eb-9d00-68798c724ace None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:44.498786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de5db7b-c840-41eb-9d00-68798c724ace None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-2de5db7b-c840-41eb-9d00-68798c724ace None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:44.513204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:44.514307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:44.515221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:44.515221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:44.515221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:44.515221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:44.515221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:44.515221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:44.515221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:44.515221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:44.515221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 475/909] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:44.515742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5db7b-c840-41eb-9d00-68798c724ace None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:44.539349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0699a2-2a92-4428-b7f8-12af58bcd41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:44.540043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0699a2-2a92-4428-b7f8-12af58bcd41f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:44.540330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0699a2-2a92-4428-b7f8-12af58bcd41f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:44.540617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0699a2-2a92-4428-b7f8-12af58bcd41f None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:44.541393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0699a2-2a92-4428-b7f8-12af58bcd41f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:44.541716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0699a2-2a92-4428-b7f8-12af58bcd41f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:44.543135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0699a2-2a92-4428-b7f8-12af58bcd41f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:44.556015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 435/910] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 23 11:42:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:44.556015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0699a2-2a92-4428-b7f8-12af58bcd41f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:44.639525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de5db7b-c840-41eb-9d00-68798c724ace None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:44.681261 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e850c9a-7dfe-47a5-9b96-9ed0b0dd9c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:44.681481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0699a2-2a92-4428-b7f8-12af58bcd41f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:44.682016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e850c9a-7dfe-47a5-9b96-9ed0b0dd9c1f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:44.682366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e850c9a-7dfe-47a5-9b96-9ed0b0dd9c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:44.682715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e850c9a-7dfe-47a5-9b96-9ed0b0dd9c1f None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:44.683429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e850c9a-7dfe-47a5-9b96-9ed0b0dd9c1f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:44.683749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e850c9a-7dfe-47a5-9b96-9ed0b0dd9c1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:44.685251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e850c9a-7dfe-47a5-9b96-9ed0b0dd9c1f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:44.691054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 476/911] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:42:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:44.691760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e850c9a-7dfe-47a5-9b96-9ed0b0dd9c1f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:44.722761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1231575-c666-4487-8f06-469bc699c69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:44.723682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1231575-c666-4487-8f06-469bc699c69f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:44.724141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1231575-c666-4487-8f06-469bc699c69f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:44.724414 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1231575-c666-4487-8f06-469bc699c69f None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:44.725240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1231575-c666-4487-8f06-469bc699c69f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:44.725553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1231575-c666-4487-8f06-469bc699c69f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:44.727023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1231575-c666-4487-8f06-469bc699c69f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:44.740595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 436/912] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:44.741845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1231575-c666-4487-8f06-469bc699c69f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:44.752860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:44.765741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:44.809269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e850c9a-7dfe-47a5-9b96-9ed0b0dd9c1f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:44.848561 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6a2c2-1461-4056-93f6-f7effa18d755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:44.849278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6a2c2-1461-4056-93f6-f7effa18d755 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:44.849564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6a2c2-1461-4056-93f6-f7effa18d755 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:44.849845 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6a2c2-1461-4056-93f6-f7effa18d755 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:44.850555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6a2c2-1461-4056-93f6-f7effa18d755 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:44.850861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6a2c2-1461-4056-93f6-f7effa18d755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:44.852232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6a2c2-1461-4056-93f6-f7effa18d755 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:44.858251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 477/913] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 23 11:42:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:44.859278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6a2c2-1461-4056-93f6-f7effa18d755 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:44.865871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1231575-c666-4487-8f06-469bc699c69f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:44.903333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87eae3-1fb6-403b-b06b-c24a21b27da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:44.904001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87eae3-1fb6-403b-b06b-c24a21b27da8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:44.904552 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87eae3-1fb6-403b-b06b-c24a21b27da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:44.904750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87eae3-1fb6-403b-b06b-c24a21b27da8 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:44.905327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87eae3-1fb6-403b-b06b-c24a21b27da8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:44.905640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87eae3-1fb6-403b-b06b-c24a21b27da8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:44.907011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87eae3-1fb6-403b-b06b-c24a21b27da8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:44.917168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 437/914] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 23 11:42:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:44.917731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87eae3-1fb6-403b-b06b-c24a21b27da8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:44.967501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6a2c2-1461-4056-93f6-f7effa18d755 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:44.981425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:45.001662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:45.003999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781120a-ebdc-4941-8a67-329ac107f3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:45.004796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781120a-ebdc-4941-8a67-329ac107f3df None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:45.005022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781120a-ebdc-4941-8a67-329ac107f3df None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:45.005305 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781120a-ebdc-4941-8a67-329ac107f3df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:45.006141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781120a-ebdc-4941-8a67-329ac107f3df None admin] RBAC: Adding query filter params (role.id=370db8aebcaf44e995d905fdd73e0bbd, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=646af794697941d6a46d76086e7e8ba9) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:45.006461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781120a-ebdc-4941-8a67-329ac107f3df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:45.007880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781120a-ebdc-4941-8a67-329ac107f3df None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:45.014577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 478/915] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:44 2022] GET /identity/v3/role_assignments?user.id=646af794697941d6a46d76086e7e8ba9&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=370db8aebcaf44e995d905fdd73e0bbd => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:45.015173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781120a-ebdc-4941-8a67-329ac107f3df None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:45.048261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87eae3-1fb6-403b-b06b-c24a21b27da8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:45.098100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0974be9-4410-4f0a-a9c0-f47419af7e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:45.098778 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0974be9-4410-4f0a-a9c0-f47419af7e34 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:45.099066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0974be9-4410-4f0a-a9c0-f47419af7e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:45.099347 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0974be9-4410-4f0a-a9c0-f47419af7e34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:45.100152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0974be9-4410-4f0a-a9c0-f47419af7e34 None admin] RBAC: Adding query filter params (interface=public, service_id=21cb1748502c48e9b7783b5b41d94edc, region_id=RegionOne) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:45.100471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0974be9-4410-4f0a-a9c0-f47419af7e34 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:45.102220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0974be9-4410-4f0a-a9c0-f47419af7e34 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:45.108444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 438/916] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 23 11:42:44 2022] GET /identity/v3/endpoints?service_id=21cb1748502c48e9b7783b5b41d94edc&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:45.114675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0974be9-4410-4f0a-a9c0-f47419af7e34 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:45.161271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781120a-ebdc-4941-8a67-329ac107f3df None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:45.221424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994425f-9b3b-485a-a6a8-a2613b477bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:45.222740 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994425f-9b3b-485a-a6a8-a2613b477bc3 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:45.223099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994425f-9b3b-485a-a6a8-a2613b477bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:45.223455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994425f-9b3b-485a-a6a8-a2613b477bc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:45.224579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994425f-9b3b-485a-a6a8-a2613b477bc3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:45.226911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994425f-9b3b-485a-a6a8-a2613b477bc3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:45.237337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0974be9-4410-4f0a-a9c0-f47419af7e34 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:45.261616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 479/917] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 23 11:42:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:45.293246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f311b3-6853-42ef-86db-26870515cc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:45.294175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f311b3-6853-42ef-86db-26870515cc14 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:45.294636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f311b3-6853-42ef-86db-26870515cc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:45.295588 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f311b3-6853-42ef-86db-26870515cc14 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:45.296451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f311b3-6853-42ef-86db-26870515cc14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:45.298162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f311b3-6853-42ef-86db-26870515cc14 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:45.309260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 439/918] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 23 11:42:45 2022] GET /identity/v3/services => generated 1497 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:42:45.757085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848d8b7-0c39-4928-8701-84ba5f02b0b9 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:45.757600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848d8b7-0c39-4928-8701-84ba5f02b0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:45.757707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5848d8b7-0c39-4928-8701-84ba5f02b0b9 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:45.759315 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 480/919] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:45.764806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dcbfe-9914-4eb1-aed8-a961024dc0cf None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:45.765094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dcbfe-9914-4eb1-aed8-a961024dc0cf None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:45.765369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645dcbfe-9914-4eb1-aed8-a961024dc0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:45.770696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:45.784887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-645dcbfe-9914-4eb1-aed8-a961024dc0cf None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:45.786116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-645dcbfe-9914-4eb1-aed8-a961024dc0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:45.857552 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 440/920] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:45.880194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848d8b7-0c39-4928-8701-84ba5f02b0b9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:45.912105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:45.986234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5848d8b7-0c39-4928-8701-84ba5f02b0b9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:46.021962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06b651-55a6-4df0-9786-5bfae0d5314d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:46.022777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06b651-55a6-4df0-9786-5bfae0d5314d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:46.023065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06b651-55a6-4df0-9786-5bfae0d5314d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:46.023348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06b651-55a6-4df0-9786-5bfae0d5314d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:46.024010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06b651-55a6-4df0-9786-5bfae0d5314d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:46.025422 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06b651-55a6-4df0-9786-5bfae0d5314d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4a06b651-55a6-4df0-9786-5bfae0d5314d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:46.038331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:46.039371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:46.040358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 481/921] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 23 11:42:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:46.046309 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dcbfe-9914-4eb1-aed8-a961024dc0cf None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:46.153845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645dcbfe-9914-4eb1-aed8-a961024dc0cf None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:46.184723 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:46.191056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39aaccc-54cc-4d26-9358-ec603e9cdff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:46.191729 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39aaccc-54cc-4d26-9358-ec603e9cdff5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:46.192019 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39aaccc-54cc-4d26-9358-ec603e9cdff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:46.192300 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39aaccc-54cc-4d26-9358-ec603e9cdff5 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:46.193045 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39aaccc-54cc-4d26-9358-ec603e9cdff5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:46.193360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39aaccc-54cc-4d26-9358-ec603e9cdff5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:46.194728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39aaccc-54cc-4d26-9358-ec603e9cdff5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:46.200199 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 441/922] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 23 11:42:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:46.209767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06b651-55a6-4df0-9786-5bfae0d5314d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:46.311257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:46.317618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06b651-55a6-4df0-9786-5bfae0d5314d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:46.355539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfeab4-8c73-4a86-b31a-d1e8897990fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:46.356607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfeab4-8c73-4a86-b31a-d1e8897990fa None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:46.357134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfeab4-8c73-4a86-b31a-d1e8897990fa None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:46.362092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfeab4-8c73-4a86-b31a-d1e8897990fa None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:46.363053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfeab4-8c73-4a86-b31a-d1e8897990fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:46.363604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfeab4-8c73-4a86-b31a-d1e8897990fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:46.365404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfeab4-8c73-4a86-b31a-d1e8897990fa None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:46.372877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 482/923] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 23 11:42:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:46.380769 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39aaccc-54cc-4d26-9358-ec603e9cdff5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:46.497705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39aaccc-54cc-4d26-9358-ec603e9cdff5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:46.535978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d54507-9e60-4290-9d92-cd6f55247561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:46.537014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d54507-9e60-4290-9d92-cd6f55247561 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:46.537509 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d54507-9e60-4290-9d92-cd6f55247561 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:46.537990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d54507-9e60-4290-9d92-cd6f55247561 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:46.538983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d54507-9e60-4290-9d92-cd6f55247561 None admin] RBAC: Adding query filter params (interface=public, service_id=b6941b21431e45f6b5fd29c497d2d9c5, region_id=RegionOne) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:46.539606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d54507-9e60-4290-9d92-cd6f55247561 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:46.541213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d54507-9e60-4290-9d92-cd6f55247561 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:46.547225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 442/924] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 23 11:42:46 2022] GET /identity/v3/endpoints?service_id=b6941b21431e45f6b5fd29c497d2d9c5&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:46.554731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfeab4-8c73-4a86-b31a-d1e8897990fa None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:46.669595 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfeab4-8c73-4a86-b31a-d1e8897990fa None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:46.706399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcb6c5-20e3-429d-adbc-bfdeded90a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:46.707070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcb6c5-20e3-429d-adbc-bfdeded90a38 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:46.707361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcb6c5-20e3-429d-adbc-bfdeded90a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:46.707642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcb6c5-20e3-429d-adbc-bfdeded90a38 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:46.708329 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcb6c5-20e3-429d-adbc-bfdeded90a38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:46.709733 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcb6c5-20e3-429d-adbc-bfdeded90a38 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:46.716154 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 483/925] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 23 11:42:46 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:42:47.254493 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73de5d3-c0a9-4cee-b41a-10ffef9cd056 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:47.255011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73de5d3-c0a9-4cee-b41a-10ffef9cd056 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:47.255126 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73de5d3-c0a9-4cee-b41a-10ffef9cd056 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:47.256686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 443/926] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:47.262134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588d94d-03f3-488f-8499-98cd193f7319 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:47.262560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588d94d-03f3-488f-8499-98cd193f7319 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:47.262734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588d94d-03f3-488f-8499-98cd193f7319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:47.281204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9588d94d-03f3-488f-8499-98cd193f7319 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:47.282422 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9588d94d-03f3-488f-8499-98cd193f7319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:47.301621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:47.355093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 484/927] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:47.379204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73de5d3-c0a9-4cee-b41a-10ffef9cd056 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:47.419499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fee7e-ab99-4f97-8bab-7c70ae113f75 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:47.419842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fee7e-ab99-4f97-8bab-7c70ae113f75 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:47.420951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fee7e-ab99-4f97-8bab-7c70ae113f75 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:47.422312 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 485/928] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:42:47.430589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18954f-424d-472f-8d18-844abace56c2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:47.430842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18954f-424d-472f-8d18-844abace56c2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:47.431131 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd18954f-424d-472f-8d18-844abace56c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:47.447581 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:47.447953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bd18954f-424d-472f-8d18-844abace56c2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:47.449266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bd18954f-424d-472f-8d18-844abace56c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:47.493493 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73de5d3-c0a9-4cee-b41a-10ffef9cd056 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:47.519266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 486/929] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:47.521007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc3283-a665-484e-bced-ab291e593683 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:47.521500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc3283-a665-484e-bced-ab291e593683 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:47.521976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc3283-a665-484e-bced-ab291e593683 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:47.523694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 487/930] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:47.528522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34223a-2c5b-4697-b98b-73812a2ced9c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:47.529075 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34223a-2c5b-4697-b98b-73812a2ced9c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:47.530149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34223a-2c5b-4697-b98b-73812a2ced9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:47.545686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ffb25-8b43-4752-a050-729cca81aeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:47.546549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ffb25-8b43-4752-a050-729cca81aeaa None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:47.546834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ffb25-8b43-4752-a050-729cca81aeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:47.547157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ffb25-8b43-4752-a050-729cca81aeaa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:47.547257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6c34223a-2c5b-4697-b98b-73812a2ced9c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:47.548368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6c34223a-2c5b-4697-b98b-73812a2ced9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:47.562222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ffb25-8b43-4752-a050-729cca81aeaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-843ffb25-8b43-4752-a050-729cca81aeaa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:47.573475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:47.574425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:47.575313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:47.575313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:47.575313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:47.575313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:47.575313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:47.575313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:47.575313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:47.575313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:47.575313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:47.575313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:47.575313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:42:47.575313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:47.577227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 444/931] 10.222.0.67 () {56 vars in 1120 bytes} [Thu Jun 23 11:42:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:47.578054 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ffb25-8b43-4752-a050-729cca81aeaa None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:47.623540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 488/932] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:42:47.624496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34223a-2c5b-4697-b98b-73812a2ced9c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:47.642673 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:47.698881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ffb25-8b43-4752-a050-729cca81aeaa None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:47.721387 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:47.733026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34223a-2c5b-4697-b98b-73812a2ced9c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:47.736844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106aea2-f916-4812-a7a7-50f39b7e6091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:47.737599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106aea2-f916-4812-a7a7-50f39b7e6091 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:47.737958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106aea2-f916-4812-a7a7-50f39b7e6091 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:47.738308 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106aea2-f916-4812-a7a7-50f39b7e6091 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:47.739127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106aea2-f916-4812-a7a7-50f39b7e6091 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:47.740639 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a106aea2-f916-4812-a7a7-50f39b7e6091 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:47.742112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: WARNING keystone.server.flask.common [None req-a106aea2-f916-4812-a7a7-50f39b7e6091 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 11:42:47.762408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.sql.core [None req-a106aea2-f916-4812-a7a7-50f39b7e6091 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 11:42:47.762408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 11:42:47.762408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [parameters: {'id': 'c8f5009a4fdf4d83a1ee7d3dd8b86790', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 11:42:47.762408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80244) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 11:42:47.769600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01b296-ad6f-44d0-8ef4-59a62a5410d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:47.770215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01b296-ad6f-44d0-8ef4-59a62a5410d2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:47.770552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01b296-ad6f-44d0-8ef4-59a62a5410d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:47.770829 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01b296-ad6f-44d0-8ef4-59a62a5410d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:47.771574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01b296-ad6f-44d0-8ef4-59a62a5410d2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:47.771883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01b296-ad6f-44d0-8ef4-59a62a5410d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:47.773320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01b296-ad6f-44d0-8ef4-59a62a5410d2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-a106aea2-f916-4812-a7a7-50f39b7e6091 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c8f5009a4fdf4d83a1ee7d3dd8b86790.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c8f5009a4fdf4d83a1ee7d3dd8b86790. Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application conn.query(q) Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 11:42:47.781180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application result.read() Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 23 11:42:47.782021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application next(self.gen) Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application yield resource Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application next(self.gen) Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise self.value Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application session.commit() Jun 23 11:42:47.783246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self.session.flush() Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application compat.raise_( Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 11:42:47.784173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 11:42:47.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application conn.query(q) Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application result.read() Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 11:42:47.788718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [parameters: {'id': 'c8f5009a4fdf4d83a1ee7d3dd8b86790', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:47.791088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:47.793680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:47.794814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:47.794814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 23 11:42:47.794814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 23 11:42:47.794814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:47.794814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:47.794814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 23 11:42:47.794814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 23 11:42:47.794814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c8f5009a4fdf4d83a1ee7d3dd8b86790. Jun 23 11:42:47.794814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:47.794814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 445/934] 10.222.0.67 () {60 vars in 1140 bytes} [Thu Jun 23 11:42:47 2022] POST /identity/v3/projects => generated 262 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:47.795529 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 489/934] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 23 11:42:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:47.795529 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01b296-ad6f-44d0-8ef4-59a62a5410d2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:47.795706 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106aea2-f916-4812-a7a7-50f39b7e6091 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:47.878176 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:47.902159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106aea2-f916-4812-a7a7-50f39b7e6091 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:47.904029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01b296-ad6f-44d0-8ef4-59a62a5410d2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:47.939078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9e82c-5d49-404a-a58b-eef4034e9c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:47.939896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e82c-5d49-404a-a58b-eef4034e9c70 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:47.940178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e82c-5d49-404a-a58b-eef4034e9c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:47.940457 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9e82c-5d49-404a-a58b-eef4034e9c70 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:47.941189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9e82c-5d49-404a-a58b-eef4034e9c70 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:47.941356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dddf3b-83e2-4c09-bae6-d5b29e5c6419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:47.942111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dddf3b-83e2-4c09-bae6-d5b29e5c6419 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:47.942401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dddf3b-83e2-4c09-bae6-d5b29e5c6419 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:47.942684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dddf3b-83e2-4c09-bae6-d5b29e5c6419 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:47.942789 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9e82c-5d49-404a-a58b-eef4034e9c70 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:47.949876 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dddf3b-83e2-4c09-bae6-d5b29e5c6419 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:47.951861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dddf3b-83e2-4c09-bae6-d5b29e5c6419 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-7ed9e82c-5d49-404a-a58b-eef4034e9c70 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:47.955907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:47.957176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 11:42:47.958048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:47.960995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 446/935] 10.222.0.67 () {56 vars in 1132 bytes} [Thu Jun 23 11:42:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:47.962051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9e82c-5d49-404a-a58b-eef4034e9c70 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-d4dddf3b-83e2-4c09-bae6-d5b29e5c6419 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:47.967486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:47.968589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:47.969545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:47.969545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:47.969545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:47.969545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:47.969545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:47.969545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:47.969545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:47.969545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:47.969545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 490/936] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:47.969545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dddf3b-83e2-4c09-bae6-d5b29e5c6419 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:48.013737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:48.072873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dddf3b-83e2-4c09-bae6-d5b29e5c6419 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:48.077438 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9e82c-5d49-404a-a58b-eef4034e9c70 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:48.109057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90178a96-8c7e-46d4-80fb-9d2cb885e3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:48.109721 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90178a96-8c7e-46d4-80fb-9d2cb885e3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:48.110011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90178a96-8c7e-46d4-80fb-9d2cb885e3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:48.110292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90178a96-8c7e-46d4-80fb-9d2cb885e3c2 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:48.110999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90178a96-8c7e-46d4-80fb-9d2cb885e3c2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:48.111312 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90178a96-8c7e-46d4-80fb-9d2cb885e3c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:48.112701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90178a96-8c7e-46d4-80fb-9d2cb885e3c2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:48.114777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:48.118522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 491/937] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 23 11:42:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:48.119572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90178a96-8c7e-46d4-80fb-9d2cb885e3c2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:48.121767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b0758-8c9d-473b-928d-6925e349400b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:48.123577 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0758-8c9d-473b-928d-6925e349400b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:48.125037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0758-8c9d-473b-928d-6925e349400b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:48.125470 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b0758-8c9d-473b-928d-6925e349400b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:48.140303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b0758-8c9d-473b-928d-6925e349400b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:48.142014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b0758-8c9d-473b-928d-6925e349400b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f67b0758-8c9d-473b-928d-6925e349400b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:48.160521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:48.161481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:48.162384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:48.162384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:48.162384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:48.162384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:48.165039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 447/938] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:48.165637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b0758-8c9d-473b-928d-6925e349400b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:48.230265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90178a96-8c7e-46d4-80fb-9d2cb885e3c2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:48.268107 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:48.268617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a54fcf-e61d-46ad-878a-02ba7eafced3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:48.269255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a54fcf-e61d-46ad-878a-02ba7eafced3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:48.269554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a54fcf-e61d-46ad-878a-02ba7eafced3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:48.269840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a54fcf-e61d-46ad-878a-02ba7eafced3 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:48.270617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a54fcf-e61d-46ad-878a-02ba7eafced3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:48.270918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a54fcf-e61d-46ad-878a-02ba7eafced3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:48.272636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a54fcf-e61d-46ad-878a-02ba7eafced3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:48.278510 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b0758-8c9d-473b-928d-6925e349400b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:48.290549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 492/939] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:48 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:48.291406 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a54fcf-e61d-46ad-878a-02ba7eafced3 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:48.329260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de38a7-ac60-47ed-bc93-95676f78542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:48.329866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de38a7-ac60-47ed-bc93-95676f78542b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:48.330156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de38a7-ac60-47ed-bc93-95676f78542b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:48.330440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de38a7-ac60-47ed-bc93-95676f78542b None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:48.331179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de38a7-ac60-47ed-bc93-95676f78542b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:48.331491 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de38a7-ac60-47ed-bc93-95676f78542b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:48.333069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de38a7-ac60-47ed-bc93-95676f78542b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:48.347329 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 448/940] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 23 11:42:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:48.348411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de38a7-ac60-47ed-bc93-95676f78542b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:48.362610 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:48.409082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a54fcf-e61d-46ad-878a-02ba7eafced3 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:48.448337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e463a8-202e-4732-a677-34cd212af7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:48.449155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e463a8-202e-4732-a677-34cd212af7ad None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:48.449525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e463a8-202e-4732-a677-34cd212af7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:48.449886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e463a8-202e-4732-a677-34cd212af7ad None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:48.450711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e463a8-202e-4732-a677-34cd212af7ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:48.451028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e463a8-202e-4732-a677-34cd212af7ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:48.452581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e463a8-202e-4732-a677-34cd212af7ad None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:48.472806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 493/941] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:48.473779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e463a8-202e-4732-a677-34cd212af7ad None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:48.481247 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de38a7-ac60-47ed-bc93-95676f78542b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:48.523988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fde3e9-5192-470f-81a8-1ad24db727c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:48.524650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fde3e9-5192-470f-81a8-1ad24db727c5 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:48.524971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fde3e9-5192-470f-81a8-1ad24db727c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:48.525688 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fde3e9-5192-470f-81a8-1ad24db727c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:48.526123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fde3e9-5192-470f-81a8-1ad24db727c5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:48.527492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fde3e9-5192-470f-81a8-1ad24db727c5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:48.565544 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 449/942] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 23 11:42:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:48.601768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e463a8-202e-4732-a677-34cd212af7ad None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:48.629031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:48.653203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574e4a7-f5e9-406a-b629-a158bed260d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:48.654045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574e4a7-f5e9-406a-b629-a158bed260d3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:48.654870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574e4a7-f5e9-406a-b629-a158bed260d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:48.655293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574e4a7-f5e9-406a-b629-a158bed260d3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:48.668359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574e4a7-f5e9-406a-b629-a158bed260d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:48.669931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574e4a7-f5e9-406a-b629-a158bed260d3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-9574e4a7-f5e9-406a-b629-a158bed260d3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:48.688167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:48.689269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:48.690220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:48.690220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:48.690220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:48.690220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:48.690220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:48.690220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:48.690220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:48.690220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:48.690220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:42:48.690220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:48.690220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 494/943] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 23 11:42:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:48.697463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fde3e9-5192-470f-81a8-1ad24db727c5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:48.714870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:48.797349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b6ffb-3396-465b-b0d4-51b483edd37a None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:48.797678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b6ffb-3396-465b-b0d4-51b483edd37a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:48.797912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b6ffb-3396-465b-b0d4-51b483edd37a None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:48.801035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 495/944] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:42:48.805660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5067e5-f2c8-41aa-a8ea-a8ba87801b9b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:48.806210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5067e5-f2c8-41aa-a8ea-a8ba87801b9b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:48.806608 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5067e5-f2c8-41aa-a8ea-a8ba87801b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:48.817104 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fde3e9-5192-470f-81a8-1ad24db727c5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:48.832336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8a5067e5-f2c8-41aa-a8ea-a8ba87801b9b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:48.833920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8a5067e5-f2c8-41aa-a8ea-a8ba87801b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:48.863241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc2eb6-f3a3-42f4-87da-737122d61494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:48.864897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc2eb6-f3a3-42f4-87da-737122d61494 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:48.865343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc2eb6-f3a3-42f4-87da-737122d61494 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:48.866083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc2eb6-f3a3-42f4-87da-737122d61494 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:48.867559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc2eb6-f3a3-42f4-87da-737122d61494 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:48.868321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc2eb6-f3a3-42f4-87da-737122d61494 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:48.870965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc2eb6-f3a3-42f4-87da-737122d61494 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:48.900152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 450/945] 10.222.0.67 () {56 vars in 1149 bytes} [Thu Jun 23 11:42:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:48.901463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc2eb6-f3a3-42f4-87da-737122d61494 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:48.917128 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 496/946] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:48.942409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5067e5-f2c8-41aa-a8ea-a8ba87801b9b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:48.999111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:49.041224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc2eb6-f3a3-42f4-87da-737122d61494 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:49.056583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5067e5-f2c8-41aa-a8ea-a8ba87801b9b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:49.068624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:49.094635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6377d8a-dff8-44e7-a6b3-0a8c527fa7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:49.095801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6377d8a-dff8-44e7-a6b3-0a8c527fa7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:49.096156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6377d8a-dff8-44e7-a6b3-0a8c527fa7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:49.096501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6377d8a-dff8-44e7-a6b3-0a8c527fa7e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:49.100237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e941f7-3734-42c5-8f81-a19e2c9342cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:49.101461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e941f7-3734-42c5-8f81-a19e2c9342cd None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:49.101932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e941f7-3734-42c5-8f81-a19e2c9342cd None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:49.102390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e941f7-3734-42c5-8f81-a19e2c9342cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:49.103226 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e941f7-3734-42c5-8f81-a19e2c9342cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:49.104853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e941f7-3734-42c5-8f81-a19e2c9342cd None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:49.108521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6377d8a-dff8-44e7-a6b3-0a8c527fa7e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:49.110340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6377d8a-dff8-44e7-a6b3-0a8c527fa7e8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-26e941f7-3734-42c5-8f81-a19e2c9342cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:49.118675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:49.119683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:49.120546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:49.123535 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 497/947] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 23 11:42:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:49.129376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e941f7-3734-42c5-8f81-a19e2c9342cd None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-c6377d8a-dff8-44e7-a6b3-0a8c527fa7e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:49.131075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:49.132142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:49.133456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:49.133456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:49.133456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:49.133456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:49.133456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 451/948] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:49.141642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6377d8a-dff8-44e7-a6b3-0a8c527fa7e8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:49.245181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e941f7-3734-42c5-8f81-a19e2c9342cd None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:49.274261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6377d8a-dff8-44e7-a6b3-0a8c527fa7e8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:49.290541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b3ace-e15b-4664-a13f-2d09ac0c3711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:49.291994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b3ace-e15b-4664-a13f-2d09ac0c3711 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:49.292446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b3ace-e15b-4664-a13f-2d09ac0c3711 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:49.293116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b3ace-e15b-4664-a13f-2d09ac0c3711 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:49.294374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b3ace-e15b-4664-a13f-2d09ac0c3711 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:49.294829 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b3ace-e15b-4664-a13f-2d09ac0c3711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:49.296891 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b3ace-e15b-4664-a13f-2d09ac0c3711 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:49.300495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:49.309990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 498/949] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 23 11:42:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:49.312243 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b3ace-e15b-4664-a13f-2d09ac0c3711 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:49.320907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a20745-abac-447b-9ea1-c2d0a57f9fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:49.321723 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a20745-abac-447b-9ea1-c2d0a57f9fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:49.322116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a20745-abac-447b-9ea1-c2d0a57f9fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:49.322501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a20745-abac-447b-9ea1-c2d0a57f9fa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:49.323341 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a20745-abac-447b-9ea1-c2d0a57f9fa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:49.323757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a20745-abac-447b-9ea1-c2d0a57f9fa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:49.325395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a20745-abac-447b-9ea1-c2d0a57f9fa4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:49.344993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 452/950] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:49.347749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a20745-abac-447b-9ea1-c2d0a57f9fa4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:49.360076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:49.451836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b3ace-e15b-4664-a13f-2d09ac0c3711 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:49.469764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a20745-abac-447b-9ea1-c2d0a57f9fa4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:49.503242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f60ef-c872-497c-b7c4-517d45e70fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:49.504341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f60ef-c872-497c-b7c4-517d45e70fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:49.504883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f60ef-c872-497c-b7c4-517d45e70fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:49.505414 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f60ef-c872-497c-b7c4-517d45e70fd2 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:49.506412 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f60ef-c872-497c-b7c4-517d45e70fd2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:49.506851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938723d-d619-4b57-8a57-9bf862e4ce04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:49.507024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f60ef-c872-497c-b7c4-517d45e70fd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:49.507837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938723d-d619-4b57-8a57-9bf862e4ce04 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:49.508223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938723d-d619-4b57-8a57-9bf862e4ce04 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:49.508603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938723d-d619-4b57-8a57-9bf862e4ce04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:49.509023 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f60ef-c872-497c-b7c4-517d45e70fd2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:49.519858 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938723d-d619-4b57-8a57-9bf862e4ce04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:49.519858 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938723d-d619-4b57-8a57-9bf862e4ce04 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:49.520615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 499/951] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 23 11:42:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:49.522330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f60ef-c872-497c-b7c4-517d45e70fd2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-c938723d-d619-4b57-8a57-9bf862e4ce04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:49.536481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:49.537629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:49.538721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:49.538721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:49.538721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:49.538721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:49.538721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:49.538721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:49.538721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:49.538721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:49.544765 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 453/952] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:49.547099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938723d-d619-4b57-8a57-9bf862e4ce04 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:49.598936 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:49.659072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:49.663581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f60ef-c872-497c-b7c4-517d45e70fd2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:49.667613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938723d-d619-4b57-8a57-9bf862e4ce04 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:49.700934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d34e8c-259e-4a5f-8885-33d5caa906e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:49.702985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d34e8c-259e-4a5f-8885-33d5caa906e8 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:49.703582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d34e8c-259e-4a5f-8885-33d5caa906e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:49.704009 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d34e8c-259e-4a5f-8885-33d5caa906e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:49.704989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df558b7-8077-4d0a-a9d0-9d2d23f3a7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:49.705189 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d34e8c-259e-4a5f-8885-33d5caa906e8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:49.705845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df558b7-8077-4d0a-a9d0-9d2d23f3a7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:49.706226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df558b7-8077-4d0a-a9d0-9d2d23f3a7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:49.706600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df558b7-8077-4d0a-a9d0-9d2d23f3a7f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:49.707165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d34e8c-259e-4a5f-8885-33d5caa906e8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:49.707573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df558b7-8077-4d0a-a9d0-9d2d23f3a7f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:49.707982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df558b7-8077-4d0a-a9d0-9d2d23f3a7f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:49.709539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df558b7-8077-4d0a-a9d0-9d2d23f3a7f7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:49.727460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 454/953] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:49.730103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df558b7-8077-4d0a-a9d0-9d2d23f3a7f7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:49.732908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 500/954] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 23 11:42:49 2022] POST /identity/v3/endpoints => generated 349 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:42:49.862020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df558b7-8077-4d0a-a9d0-9d2d23f3a7f7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:49.863777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:49.905223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380fe6c-809d-4a32-abd5-d70ab3671e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:49.907090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380fe6c-809d-4a32-abd5-d70ab3671e92 None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:49.907090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380fe6c-809d-4a32-abd5-d70ab3671e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:49.907090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380fe6c-809d-4a32-abd5-d70ab3671e92 None admin] PATH_INFO: `/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/646af794697941d6a46d76086e7e8ba9/roles/370db8aebcaf44e995d905fdd73e0bbd` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:49.909250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380fe6c-809d-4a32-abd5-d70ab3671e92 None admin] RBAC: Authorizing `identity:create_grant(project_id=19c2f661cc934aa7869e9b3d4efa875e, user_id=646af794697941d6a46d76086e7e8ba9, role_id=370db8aebcaf44e995d905fdd73e0bbd)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:49.910613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380fe6c-809d-4a32-abd5-d70ab3671e92 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:49.930024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 455/955] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:42:49 2022] PUT /identity/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/646af794697941d6a46d76086e7e8ba9/roles/370db8aebcaf44e995d905fdd73e0bbd => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:42:50.574460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb199d-4630-415d-b12c-c2ee7c1b0399 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:50.574980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb199d-4630-415d-b12c-c2ee7c1b0399 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:50.575094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb199d-4630-415d-b12c-c2ee7c1b0399 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:50.576647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 501/956] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:50.582146 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73823f76-7561-42eb-9172-9dab5bcf8cde None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:50.582564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73823f76-7561-42eb-9172-9dab5bcf8cde None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:50.582760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73823f76-7561-42eb-9172-9dab5bcf8cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:50.601607 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-73823f76-7561-42eb-9172-9dab5bcf8cde None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:50.603143 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-73823f76-7561-42eb-9172-9dab5bcf8cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:50.681270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 456/957] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:50.706065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb199d-4630-415d-b12c-c2ee7c1b0399 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:50.769410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410da76-ebff-4018-a2b4-0775f5b1d75b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:50.769749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410da76-ebff-4018-a2b4-0775f5b1d75b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:50.769982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410da76-ebff-4018-a2b4-0775f5b1d75b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:50.771372 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 457/958] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:42:50.776273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741258bf-fed5-4d09-b6a1-d702d68bf0de None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:50.776566 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741258bf-fed5-4d09-b6a1-d702d68bf0de None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:50.776898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741258bf-fed5-4d09-b6a1-d702d68bf0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:50.795903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-741258bf-fed5-4d09-b6a1-d702d68bf0de None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:50.797327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-741258bf-fed5-4d09-b6a1-d702d68bf0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:50.817731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb199d-4630-415d-b12c-c2ee7c1b0399 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:50.869932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b386db-83b1-4cdd-b402-6778c233df75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:50.870565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b386db-83b1-4cdd-b402-6778c233df75 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:50.871285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b386db-83b1-4cdd-b402-6778c233df75 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:50.871568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b386db-83b1-4cdd-b402-6778c233df75 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:50.872555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b386db-83b1-4cdd-b402-6778c233df75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:50.874507 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b386db-83b1-4cdd-b402-6778c233df75 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:50.876094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: WARNING keystone.server.flask.common [None req-e5b386db-83b1-4cdd-b402-6778c233df75 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 11:42:50.888821 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 458/959] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:42:50.897458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.sql.core [None req-e5b386db-83b1-4cdd-b402-6778c233df75 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 11:42:50.897458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 11:42:50.897458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [parameters: {'user_id': '1912ef7f8dc84bc38ab871732528e2a3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 11:42:50.897458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80243) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e5b386db-83b1-4cdd-b402-6778c233df75 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application conn.query(q) Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result.read() Jun 23 11:42:50.905763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 11:42:50.906670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application next(self.gen) Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application yield resource Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application next(self.gen) Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise self.value Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application session.commit() Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 11:42:50.908446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.session.flush() Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application compat.raise_( Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 11:42:50.910817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 11:42:50.917467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application conn.query(q) Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application result.read() Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 11:42:50.918536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [parameters: {'user_id': '1912ef7f8dc84bc38ab871732528e2a3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:50.919459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 23 11:42:50.920938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 23 11:42:50.921880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 23 11:42:50.921880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 23 11:42:50.921880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 11:42:50.921880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 11:42:50.921880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 11:42:50.921880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:50.921880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 502/960] 10.222.0.67 () {60 vars in 1131 bytes} [Thu Jun 23 11:42:50 2022] POST /identity/v3/users => generated 160 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:50.922271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741258bf-fed5-4d09-b6a1-d702d68bf0de None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:50.922355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b386db-83b1-4cdd-b402-6778c233df75 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:50.933848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:50.944925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:51.032057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741258bf-fed5-4d09-b6a1-d702d68bf0de None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:51.049862 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b386db-83b1-4cdd-b402-6778c233df75 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:51.070304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7e301-4795-4c98-87a1-368cd8c2b1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:51.071454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7e301-4795-4c98-87a1-368cd8c2b1fa None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:51.071883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7e301-4795-4c98-87a1-368cd8c2b1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:51.072313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7e301-4795-4c98-87a1-368cd8c2b1fa None admin] PATH_INFO: `/v3/services/compute` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:51.073158 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7e301-4795-4c98-87a1-368cd8c2b1fa None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:51.074881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7e301-4795-4c98-87a1-368cd8c2b1fa None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:51.086505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586f8f7-0130-4c84-9396-142db0682d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:51.087323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586f8f7-0130-4c84-9396-142db0682d7f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:51.087610 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586f8f7-0130-4c84-9396-142db0682d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:51.087892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586f8f7-0130-4c84-9396-142db0682d7f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-c8d7e301-4795-4c98-87a1-368cd8c2b1fa None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:51.089538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:51.090781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:42:51.091842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:51.094414 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 459/961] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:51.100546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7e301-4795-4c98-87a1-368cd8c2b1fa None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:51.102364 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586f8f7-0130-4c84-9396-142db0682d7f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:51.104369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586f8f7-0130-4c84-9396-142db0682d7f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4586f8f7-0130-4c84-9396-142db0682d7f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:51.124625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:51.125634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:51.126587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:51.126587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:51.126587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:51.126587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:51.126587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:51.126587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:51.126587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:51.126587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:51.126587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 11:42:51.126587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:51.131209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 503/962] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:42:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:51.132307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586f8f7-0130-4c84-9396-142db0682d7f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:51.204705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:51.229959 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7e301-4795-4c98-87a1-368cd8c2b1fa None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:51.236432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:51.266221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586f8f7-0130-4c84-9396-142db0682d7f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:51.273017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7535c0-2a02-4aa2-9fcf-448af55a5995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:51.274064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7535c0-2a02-4aa2-9fcf-448af55a5995 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:51.274554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7535c0-2a02-4aa2-9fcf-448af55a5995 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:51.275430 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7535c0-2a02-4aa2-9fcf-448af55a5995 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:51.276346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7535c0-2a02-4aa2-9fcf-448af55a5995 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:51.277250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7535c0-2a02-4aa2-9fcf-448af55a5995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:51.279452 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7535c0-2a02-4aa2-9fcf-448af55a5995 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:51.291234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 460/963] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:51.291728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7535c0-2a02-4aa2-9fcf-448af55a5995 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:51.308634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bff0f8-b12e-422f-9fa1-1c5b35e740bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:51.309415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bff0f8-b12e-422f-9fa1-1c5b35e740bd None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:51.309697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bff0f8-b12e-422f-9fa1-1c5b35e740bd None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:51.310108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bff0f8-b12e-422f-9fa1-1c5b35e740bd None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:51.311617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bff0f8-b12e-422f-9fa1-1c5b35e740bd None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:51.311928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bff0f8-b12e-422f-9fa1-1c5b35e740bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:51.313587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bff0f8-b12e-422f-9fa1-1c5b35e740bd None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:51.364562 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 504/964] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 23 11:42:51 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:51.414951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7535c0-2a02-4aa2-9fcf-448af55a5995 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:51.466194 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17b8748-0ea1-4fe8-9a8f-3756abcd1e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:51.467356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b8748-0ea1-4fe8-9a8f-3756abcd1e3c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:51.467892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b8748-0ea1-4fe8-9a8f-3756abcd1e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:51.468422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17b8748-0ea1-4fe8-9a8f-3756abcd1e3c None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:51.469417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b8748-0ea1-4fe8-9a8f-3756abcd1e3c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:51.469952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b8748-0ea1-4fe8-9a8f-3756abcd1e3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:51.471672 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17b8748-0ea1-4fe8-9a8f-3756abcd1e3c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:51.477703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 461/965] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:51.889565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b3aa4-e29e-4405-be45-28725926f470 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:51.890083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b3aa4-e29e-4405-be45-28725926f470 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:51.890186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b3aa4-e29e-4405-be45-28725926f470 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:51.891641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 505/966] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:51.896820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a57d50-cef2-4fc7-baf6-e654b32f5f01 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:51.897249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a57d50-cef2-4fc7-baf6-e654b32f5f01 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:51.897421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a57d50-cef2-4fc7-baf6-e654b32f5f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:51.914441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-59a57d50-cef2-4fc7-baf6-e654b32f5f01 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:51.915643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-59a57d50-cef2-4fc7-baf6-e654b32f5f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:51.918718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:51.986148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 462/967] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:52.009106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b3aa4-e29e-4405-be45-28725926f470 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:52.050674 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5971b8c-b326-4443-9c83-0afc3d017acb None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.050993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5971b8c-b326-4443-9c83-0afc3d017acb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.051248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5971b8c-b326-4443-9c83-0afc3d017acb None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.052834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 463/968] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:42:52.057770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519dec86-39c5-4424-9c61-8a4c93182de2 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.058067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519dec86-39c5-4424-9c61-8a4c93182de2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.058360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519dec86-39c5-4424-9c61-8a4c93182de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.078948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-519dec86-39c5-4424-9c61-8a4c93182de2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:52.080157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-519dec86-39c5-4424-9c61-8a4c93182de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:52.121833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b3aa4-e29e-4405-be45-28725926f470 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:52.154904 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 464/969] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:42:52.162568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6636989-e727-4e87-89d1-1c9b73739f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:52.163300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6636989-e727-4e87-89d1-1c9b73739f79 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.163582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6636989-e727-4e87-89d1-1c9b73739f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.163855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6636989-e727-4e87-89d1-1c9b73739f79 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.164512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6636989-e727-4e87-89d1-1c9b73739f79 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:52.165921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6636989-e727-4e87-89d1-1c9b73739f79 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:52.167867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:52.179037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519dec86-39c5-4424-9c61-8a4c93182de2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-a6636989-e727-4e87-89d1-1c9b73739f79 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:52.179621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:52.180790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:52.181649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:52.186287 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 506/970] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 23 11:42:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:52.186940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6636989-e727-4e87-89d1-1c9b73739f79 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:52.264806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:52.294359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6636989-e727-4e87-89d1-1c9b73739f79 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:52.301245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519dec86-39c5-4424-9c61-8a4c93182de2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:52.328756 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff115c-dd00-410f-b53a-8f48460af3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:52.329427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff115c-dd00-410f-b53a-8f48460af3ab None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.329719 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff115c-dd00-410f-b53a-8f48460af3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.330004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff115c-dd00-410f-b53a-8f48460af3ab None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.330712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff115c-dd00-410f-b53a-8f48460af3ab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:52.331029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff115c-dd00-410f-b53a-8f48460af3ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:52.332405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff115c-dd00-410f-b53a-8f48460af3ab None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:52.337574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 507/971] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 23 11:42:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:52.344103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff115c-dd00-410f-b53a-8f48460af3ab None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:52.345670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b27364-563d-410e-aaac-cb375d192d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:52.346947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b27364-563d-410e-aaac-cb375d192d35 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.347406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b27364-563d-410e-aaac-cb375d192d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.348003 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b27364-563d-410e-aaac-cb375d192d35 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.365284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b27364-563d-410e-aaac-cb375d192d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-e7b27364-563d-410e-aaac-cb375d192d35 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:42:52.381635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:52.382662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:52.383519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 465/972] 10.222.0.67 () {56 vars in 1120 bytes} [Thu Jun 23 11:42:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:52.391910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b27364-563d-410e-aaac-cb375d192d35 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:52.441572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:52.488495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff115c-dd00-410f-b53a-8f48460af3ab None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:52.515666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b27364-563d-410e-aaac-cb375d192d35 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:52.524452 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d0767-96a5-4b6c-8f19-564ce40d4a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:52.525076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d0767-96a5-4b6c-8f19-564ce40d4a3a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.525349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d0767-96a5-4b6c-8f19-564ce40d4a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.525651 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d0767-96a5-4b6c-8f19-564ce40d4a3a None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.526366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d0767-96a5-4b6c-8f19-564ce40d4a3a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:52.526662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d0767-96a5-4b6c-8f19-564ce40d4a3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:52.528049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d0767-96a5-4b6c-8f19-564ce40d4a3a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:52.534955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 508/973] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 23 11:42:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:52.540804 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d0767-96a5-4b6c-8f19-564ce40d4a3a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:52.554527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:52.561627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc31f4-70a3-4d8e-bfb9-b0fa7a7790bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:52.562297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc31f4-70a3-4d8e-bfb9-b0fa7a7790bf None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.562584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc31f4-70a3-4d8e-bfb9-b0fa7a7790bf None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.562863 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fc31f4-70a3-4d8e-bfb9-b0fa7a7790bf None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.564058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc31f4-70a3-4d8e-bfb9-b0fa7a7790bf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:52.564380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc31f4-70a3-4d8e-bfb9-b0fa7a7790bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:52.565809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fc31f4-70a3-4d8e-bfb9-b0fa7a7790bf None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:52.574846 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 466/974] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 23 11:42:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:52.580116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc31f4-70a3-4d8e-bfb9-b0fa7a7790bf None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:52.652004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d0767-96a5-4b6c-8f19-564ce40d4a3a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:52.692243 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fc31f4-70a3-4d8e-bfb9-b0fa7a7790bf None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:52.697029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:52.709534 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83646a7d-87bc-4604-9654-b6f45380c401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:52.710199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83646a7d-87bc-4604-9654-b6f45380c401 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.710498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83646a7d-87bc-4604-9654-b6f45380c401 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.710788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83646a7d-87bc-4604-9654-b6f45380c401 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.711602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83646a7d-87bc-4604-9654-b6f45380c401 None admin] RBAC: Adding query filter params (interface=admin, service_id=b6941b21431e45f6b5fd29c497d2d9c5, region_id=RegionOne) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:52.711914 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83646a7d-87bc-4604-9654-b6f45380c401 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:52.713317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83646a7d-87bc-4604-9654-b6f45380c401 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:52.725545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 509/975] 10.222.0.67 () {56 vars in 1249 bytes} [Thu Jun 23 11:42:52 2022] GET /identity/v3/endpoints?service_id=b6941b21431e45f6b5fd29c497d2d9c5&interface=admin®ion_id=RegionOne => generated 193 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:52.731109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83646a7d-87bc-4604-9654-b6f45380c401 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:52.735085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188fa0f-12ba-4f99-89a1-43ccde1474ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:52.735880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188fa0f-12ba-4f99-89a1-43ccde1474ef None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.736164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188fa0f-12ba-4f99-89a1-43ccde1474ef None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.736439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188fa0f-12ba-4f99-89a1-43ccde1474ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.747272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188fa0f-12ba-4f99-89a1-43ccde1474ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:52.748816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188fa0f-12ba-4f99-89a1-43ccde1474ef None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-3188fa0f-12ba-4f99-89a1-43ccde1474ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:52.766298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:52.767274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:52.768251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:52.768251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:52.768251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:52.768251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:52.768251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 467/976] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:52.773171 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188fa0f-12ba-4f99-89a1-43ccde1474ef None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:52.820540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:52.843197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83646a7d-87bc-4604-9654-b6f45380c401 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:52.881844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188fa0f-12ba-4f99-89a1-43ccde1474ef None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:52.882336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba8424-4225-49d1-a532-67fa8f3bb3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:52.883002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba8424-4225-49d1-a532-67fa8f3bb3b2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.883297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba8424-4225-49d1-a532-67fa8f3bb3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.883576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba8424-4225-49d1-a532-67fa8f3bb3b2 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.884261 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba8424-4225-49d1-a532-67fa8f3bb3b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:52.885659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba8424-4225-49d1-a532-67fa8f3bb3b2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:52.892708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 510/977] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 23 11:42:52 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:42:52.927450 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7aac4-6fd2-4911-85d8-a652135b927e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:52.928132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7aac4-6fd2-4911-85d8-a652135b927e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:52.928425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7aac4-6fd2-4911-85d8-a652135b927e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:52.928739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7aac4-6fd2-4911-85d8-a652135b927e None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:52.929494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7aac4-6fd2-4911-85d8-a652135b927e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:52.929806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7aac4-6fd2-4911-85d8-a652135b927e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:52.931208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7aac4-6fd2-4911-85d8-a652135b927e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:52.950790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 468/978] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:52.951712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba8424-4225-49d1-a532-67fa8f3bb3b2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:53.014467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:53.064996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba8424-4225-49d1-a532-67fa8f3bb3b2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:53.099118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e7c66-e94b-43a1-b391-7cf6c622268f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:53.099930 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e7c66-e94b-43a1-b391-7cf6c622268f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:53.100220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e7c66-e94b-43a1-b391-7cf6c622268f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:53.100500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e7c66-e94b-43a1-b391-7cf6c622268f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:53.108977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e7c66-e94b-43a1-b391-7cf6c622268f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:53.110421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e7c66-e94b-43a1-b391-7cf6c622268f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4b5e7c66-e94b-43a1-b391-7cf6c622268f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:42:53.125326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:53.126354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:53.127326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:53.127326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:53.127326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:42:53.127326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:42:53.127326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:42:53.127326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:42:53.127326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:42:53.127326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:42:53.127326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 11:42:53.127326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:53.127326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 511/979] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 23 11:42:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:53.134445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7aac4-6fd2-4911-85d8-a652135b927e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:53.240781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7aac4-6fd2-4911-85d8-a652135b927e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:53.243310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:53.276772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e223c72-3151-4b8b-811a-90b760bd76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:53.277446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e223c72-3151-4b8b-811a-90b760bd76c9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:53.277733 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e223c72-3151-4b8b-811a-90b760bd76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:53.278012 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e223c72-3151-4b8b-811a-90b760bd76c9 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:53.278853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e223c72-3151-4b8b-811a-90b760bd76c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:53.279168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e223c72-3151-4b8b-811a-90b760bd76c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:53.281181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e223c72-3151-4b8b-811a-90b760bd76c9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:53.299407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 469/980] 10.222.0.67 () {56 vars in 1149 bytes} [Thu Jun 23 11:42:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:53.304797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e7c66-e94b-43a1-b391-7cf6c622268f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:53.409653 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e7c66-e94b-43a1-b391-7cf6c622268f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:53.424549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:53.446215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934d206-5fe3-4337-8263-8eb77673e6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:53.446905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934d206-5fe3-4337-8263-8eb77673e6ba None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:53.447157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934d206-5fe3-4337-8263-8eb77673e6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:53.447439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3934d206-5fe3-4337-8263-8eb77673e6ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:53.517074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934d206-5fe3-4337-8263-8eb77673e6ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:53.518541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3934d206-5fe3-4337-8263-8eb77673e6ba None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:53.582542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e808c5-233c-47b0-b648-686c8e297aa9 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:53.582885 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e808c5-233c-47b0-b648-686c8e297aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:53.583124 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e808c5-233c-47b0-b648-686c8e297aa9 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:53.584652 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 470/981] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:53.589644 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d07752-0fa4-48d6-937a-6c371d5a0340 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:53.589939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d07752-0fa4-48d6-937a-6c371d5a0340 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:53.590222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d07752-0fa4-48d6-937a-6c371d5a0340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:53.607084 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c0d07752-0fa4-48d6-937a-6c371d5a0340 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:53.608365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c0d07752-0fa4-48d6-937a-6c371d5a0340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:53.625385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 512/982] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:42:53 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:42:53.626708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934d206-5fe3-4337-8263-8eb77673e6ba None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:53.642300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:53.685725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 471/983] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:42:53.708735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d07752-0fa4-48d6-937a-6c371d5a0340 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:53.751366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3934d206-5fe3-4337-8263-8eb77673e6ba None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:53.786327 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5727aa-73e1-4f3c-b696-dae7c4331871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:53.787023 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5727aa-73e1-4f3c-b696-dae7c4331871 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:53.787312 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5727aa-73e1-4f3c-b696-dae7c4331871 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:53.787597 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5727aa-73e1-4f3c-b696-dae7c4331871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:53.805067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5727aa-73e1-4f3c-b696-dae7c4331871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:53.806520 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5727aa-73e1-4f3c-b696-dae7c4331871 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:53.812409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:53.822983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d07752-0fa4-48d6-937a-6c371d5a0340 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8e5727aa-73e1-4f3c-b696-dae7c4331871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:42:53.825765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:53.826717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:53.827604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:42:53.827604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:42:53.827604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:42:53.827604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:53.827604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 513/984] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:53.832447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5727aa-73e1-4f3c-b696-dae7c4331871 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:53.869357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89ebf0-57c8-4c7c-92da-ecf8885fd606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:53.870565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89ebf0-57c8-4c7c-92da-ecf8885fd606 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:53.870994 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89ebf0-57c8-4c7c-92da-ecf8885fd606 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:53.871415 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89ebf0-57c8-4c7c-92da-ecf8885fd606 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:53.872445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89ebf0-57c8-4c7c-92da-ecf8885fd606 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:53.874765 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89ebf0-57c8-4c7c-92da-ecf8885fd606 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:53.888774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 472/985] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:42:53 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:53.908048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:53.950923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5727aa-73e1-4f3c-b696-dae7c4331871 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:53.999352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e1aac-f144-4f35-ad8d-145a5e1d126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:54.000050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e1aac-f144-4f35-ad8d-145a5e1d126e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:54.000340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e1aac-f144-4f35-ad8d-145a5e1d126e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:54.000622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e1aac-f144-4f35-ad8d-145a5e1d126e None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:54.001413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e1aac-f144-4f35-ad8d-145a5e1d126e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:54.001727 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e1aac-f144-4f35-ad8d-145a5e1d126e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:54.003245 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e1aac-f144-4f35-ad8d-145a5e1d126e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:54.022409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 514/986] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:54.023354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89ebf0-57c8-4c7c-92da-ecf8885fd606 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:54.133970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89ebf0-57c8-4c7c-92da-ecf8885fd606 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:54.168451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ffdc1-b363-4b61-a3cd-fe01ffe3187e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:54.169275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ffdc1-b363-4b61-a3cd-fe01ffe3187e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:54.169562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ffdc1-b363-4b61-a3cd-fe01ffe3187e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:54.170486 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56ffdc1-b363-4b61-a3cd-fe01ffe3187e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:54.177352 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ffdc1-b363-4b61-a3cd-fe01ffe3187e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:54.178791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56ffdc1-b363-4b61-a3cd-fe01ffe3187e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d56ffdc1-b363-4b61-a3cd-fe01ffe3187e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:42:54.192291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:54.193364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:54.194241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:42:54.194241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:42:54.194241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:42:54.194241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:42:54.194241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:42:54.194241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:42:54.194241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:42:54.194241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:42:54.194241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 473/987] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:54.199445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e1aac-f144-4f35-ad8d-145a5e1d126e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:54.304786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e1aac-f144-4f35-ad8d-145a5e1d126e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:54.327346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:54.340167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e941aa-417b-47f2-9668-2ead3dc2bc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:54.341240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e941aa-417b-47f2-9668-2ead3dc2bc66 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:54.341628 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e941aa-417b-47f2-9668-2ead3dc2bc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:54.342715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e941aa-417b-47f2-9668-2ead3dc2bc66 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:54.343714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e941aa-417b-47f2-9668-2ead3dc2bc66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:54.344099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e941aa-417b-47f2-9668-2ead3dc2bc66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:54.345637 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e941aa-417b-47f2-9668-2ead3dc2bc66 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:54.358409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 515/988] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:42:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:54.364451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ffdc1-b363-4b61-a3cd-fe01ffe3187e None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:54.382844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:54.475195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56ffdc1-b363-4b61-a3cd-fe01ffe3187e None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:54.509022 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c3fa7-ff2a-4dd4-b0ee-830387d167a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:54.509673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c3fa7-ff2a-4dd4-b0ee-830387d167a2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:54.509955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c3fa7-ff2a-4dd4-b0ee-830387d167a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:54.510236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c3fa7-ff2a-4dd4-b0ee-830387d167a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:54.511068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c3fa7-ff2a-4dd4-b0ee-830387d167a2 None admin] RBAC: Adding query filter params (role.id=370db8aebcaf44e995d905fdd73e0bbd, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=646af794697941d6a46d76086e7e8ba9) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:54.511383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c3fa7-ff2a-4dd4-b0ee-830387d167a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:54.512835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c3fa7-ff2a-4dd4-b0ee-830387d167a2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:54.519467 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 474/989] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:42:54 2022] GET /identity/v3/role_assignments?user.id=646af794697941d6a46d76086e7e8ba9&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=370db8aebcaf44e995d905fdd73e0bbd => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:54.971044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b715f-b632-42be-a6ee-6af282319d29 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:54.971547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b715f-b632-42be-a6ee-6af282319d29 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:54.971627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b715f-b632-42be-a6ee-6af282319d29 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:54.973064 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 516/990] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:54.978426 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a37c4-9050-479f-9d96-8ccce73e36c4 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:54.978725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a37c4-9050-479f-9d96-8ccce73e36c4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:54.979012 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a37c4-9050-479f-9d96-8ccce73e36c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:54.995962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7a1a37c4-9050-479f-9d96-8ccce73e36c4 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:54.997189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7a1a37c4-9050-479f-9d96-8ccce73e36c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:55.072722 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 475/991] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:55.094032 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b715f-b632-42be-a6ee-6af282319d29 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:55.194803 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b715f-b632-42be-a6ee-6af282319d29 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:55.229786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccbfa7-2c2b-427e-9d9d-d2b91a407878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:55.230510 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccbfa7-2c2b-427e-9d9d-d2b91a407878 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:55.230795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccbfa7-2c2b-427e-9d9d-d2b91a407878 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:55.231072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccbfa7-2c2b-427e-9d9d-d2b91a407878 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:55.231765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccbfa7-2c2b-427e-9d9d-d2b91a407878 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:55.233178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccccbfa7-2c2b-427e-9d9d-d2b91a407878 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-ccccbfa7-2c2b-427e-9d9d-d2b91a407878 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:55.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:55.246693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:55.247555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 517/992] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 23 11:42:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:55.254299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a37c4-9050-479f-9d96-8ccce73e36c4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:55.262708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:55.356279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a37c4-9050-479f-9d96-8ccce73e36c4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:55.390960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3eefa-b4c4-4e5d-908e-02693b01d78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:55.391618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3eefa-b4c4-4e5d-908e-02693b01d78d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:55.391905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3eefa-b4c4-4e5d-908e-02693b01d78d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:55.392184 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c3eefa-b4c4-4e5d-908e-02693b01d78d None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:55.392914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3eefa-b4c4-4e5d-908e-02693b01d78d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:55.393224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3eefa-b4c4-4e5d-908e-02693b01d78d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:55.394584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c3eefa-b4c4-4e5d-908e-02693b01d78d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:55.399628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 476/993] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 23 11:42:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:55.405231 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccbfa7-2c2b-427e-9d9d-d2b91a407878 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:55.454773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:55.508620 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccbfa7-2c2b-427e-9d9d-d2b91a407878 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:55.542794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f1fd8-7098-4939-9d04-a82edf882e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:55.543461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f1fd8-7098-4939-9d04-a82edf882e12 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:55.543777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f1fd8-7098-4939-9d04-a82edf882e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:55.544062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f1fd8-7098-4939-9d04-a82edf882e12 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:55.544827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f1fd8-7098-4939-9d04-a82edf882e12 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:55.545146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f1fd8-7098-4939-9d04-a82edf882e12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:55.546505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f1fd8-7098-4939-9d04-a82edf882e12 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:55.556048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 518/994] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 23 11:42:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:55.557271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3eefa-b4c4-4e5d-908e-02693b01d78d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:55.629810 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:55.670194 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c3eefa-b4c4-4e5d-908e-02693b01d78d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:55.708222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af22c82-01f2-42fd-bb73-7c67a249de74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:55.708981 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af22c82-01f2-42fd-bb73-7c67a249de74 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:55.709279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af22c82-01f2-42fd-bb73-7c67a249de74 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:55.709558 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af22c82-01f2-42fd-bb73-7c67a249de74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:55.710304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af22c82-01f2-42fd-bb73-7c67a249de74 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:55.711672 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af22c82-01f2-42fd-bb73-7c67a249de74 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:55.732808 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 477/995] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 23 11:42:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:55.979956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b4d00-61a3-4bf2-b20f-196eb7872c43 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:55.980458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b4d00-61a3-4bf2-b20f-196eb7872c43 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:55.980537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b4d00-61a3-4bf2-b20f-196eb7872c43 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:55.982046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 519/996] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:55.987493 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969379b-a5c7-4f6e-b2d9-148f4631c7ee None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:55.987788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969379b-a5c7-4f6e-b2d9-148f4631c7ee None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:55.988069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969379b-a5c7-4f6e-b2d9-148f4631c7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:56.005214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1969379b-a5c7-4f6e-b2d9-148f4631c7ee None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:56.006396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1969379b-a5c7-4f6e-b2d9-148f4631c7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:56.084018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 478/997] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:55 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:56.106725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b4d00-61a3-4bf2-b20f-196eb7872c43 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:56.215412 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b4d00-61a3-4bf2-b20f-196eb7872c43 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:56.227170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:56.253893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c61cb4-dcfb-41d2-94c7-d6b2a2d89552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:56.254601 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c61cb4-dcfb-41d2-94c7-d6b2a2d89552 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:56.254887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c61cb4-dcfb-41d2-94c7-d6b2a2d89552 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:56.255190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c61cb4-dcfb-41d2-94c7-d6b2a2d89552 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:56.255821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c61cb4-dcfb-41d2-94c7-d6b2a2d89552 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:56.257235 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c61cb4-dcfb-41d2-94c7-d6b2a2d89552 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-05c61cb4-dcfb-41d2-94c7-d6b2a2d89552 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:56.269232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:56.270337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:56.271646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 520/998] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:56.276714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969379b-a5c7-4f6e-b2d9-148f4631c7ee None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:56.336083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:56.384720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969379b-a5c7-4f6e-b2d9-148f4631c7ee None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:56.426194 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f696194-f830-40ce-9184-03b3fde4c154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:56.426871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f696194-f830-40ce-9184-03b3fde4c154 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:56.427160 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f696194-f830-40ce-9184-03b3fde4c154 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:56.427440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f696194-f830-40ce-9184-03b3fde4c154 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:56.428186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f696194-f830-40ce-9184-03b3fde4c154 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:56.428499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f696194-f830-40ce-9184-03b3fde4c154 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:56.429901 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f696194-f830-40ce-9184-03b3fde4c154 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:56.440083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 479/999] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:56.441350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c61cb4-dcfb-41d2-94c7-d6b2a2d89552 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:56.542417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c61cb4-dcfb-41d2-94c7-d6b2a2d89552 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:56.576177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fecd24-515a-48e9-b051-0a2829d129af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:56.576856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fecd24-515a-48e9-b051-0a2829d129af None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:56.577145 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fecd24-515a-48e9-b051-0a2829d129af None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:56.577423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fecd24-515a-48e9-b051-0a2829d129af None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:56.578127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fecd24-515a-48e9-b051-0a2829d129af None admin] RBAC: Adding query filter params (type=compute) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:56.578437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fecd24-515a-48e9-b051-0a2829d129af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:56.579795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fecd24-515a-48e9-b051-0a2829d129af None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:56.585085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 521/1000] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:56.590683 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f696194-f830-40ce-9184-03b3fde4c154 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:56.622859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e578d7a-2280-4f99-a012-13dad9bb7226 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:56.623736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e578d7a-2280-4f99-a012-13dad9bb7226 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:56.624027 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e578d7a-2280-4f99-a012-13dad9bb7226 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:56.625767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 522/1001] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:42:56.630379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fe67c-2db8-4ac8-ae3f-7afe3c0c6ee0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:56.630672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fe67c-2db8-4ac8-ae3f-7afe3c0c6ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:56.631003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fe67c-2db8-4ac8-ae3f-7afe3c0c6ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:56.647260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-411fe67c-2db8-4ac8-ae3f-7afe3c0c6ee0 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:56.649013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-411fe67c-2db8-4ac8-ae3f-7afe3c0c6ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:56.686870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:56.710943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f696194-f830-40ce-9184-03b3fde4c154 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:56.732942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 523/1002] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:56 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:42:56.733368 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:56.753373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c30b59-f932-449d-95c9-efe5f7022750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:56.754439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c30b59-f932-449d-95c9-efe5f7022750 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:56.754737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c30b59-f932-449d-95c9-efe5f7022750 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:56.755022 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c30b59-f932-449d-95c9-efe5f7022750 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:56.755280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fe67c-2db8-4ac8-ae3f-7afe3c0c6ee0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:56.755950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c30b59-f932-449d-95c9-efe5f7022750 None admin] RBAC: Adding query filter params (interface=public, service_id=a03023c026da40469e52b313f79480be, region_id=RegionOne) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:56.756281 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c30b59-f932-449d-95c9-efe5f7022750 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:56.759969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c30b59-f932-449d-95c9-efe5f7022750 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:56.770937 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 480/1003] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 23 11:42:56 2022] GET /identity/v3/endpoints?service_id=a03023c026da40469e52b313f79480be&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:42:56.776097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c30b59-f932-449d-95c9-efe5f7022750 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:56.876241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fe67c-2db8-4ac8-ae3f-7afe3c0c6ee0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:56.883466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c30b59-f932-449d-95c9-efe5f7022750 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:56.915495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca6893-3b9c-44fc-9fe4-09515d79b9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:56.916395 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca6893-3b9c-44fc-9fe4-09515d79b9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:56.916739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca6893-3b9c-44fc-9fe4-09515d79b9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:56.917020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca6893-3b9c-44fc-9fe4-09515d79b9f0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:56.917671 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca6893-3b9c-44fc-9fe4-09515d79b9f0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:56.919036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ca6893-3b9c-44fc-9fe4-09515d79b9f0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:56.922438 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a561a29-686b-431d-b5fa-4e60eadda3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:56.923105 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a561a29-686b-431d-b5fa-4e60eadda3ff None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:56.923389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a561a29-686b-431d-b5fa-4e60eadda3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:56.923667 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a561a29-686b-431d-b5fa-4e60eadda3ff None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:56.924355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a561a29-686b-431d-b5fa-4e60eadda3ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:56.925813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a561a29-686b-431d-b5fa-4e60eadda3ff None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-83ca6893-3b9c-44fc-9fe4-09515d79b9f0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:56.931541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:56.933276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:56.934302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 524/1004] 10.222.0.67 () {56 vars in 1105 bytes} [Thu Jun 23 11:42:56 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:42:56.945113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 481/1005] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 23 11:42:56 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:42:56.946082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca6893-3b9c-44fc-9fe4-09515d79b9f0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:56.981068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:57.062670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca6893-3b9c-44fc-9fe4-09515d79b9f0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:57.101615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e692ac-2613-4916-9879-c60050804b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:57.102279 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e692ac-2613-4916-9879-c60050804b5e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.102563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e692ac-2613-4916-9879-c60050804b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.102839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e692ac-2613-4916-9879-c60050804b5e None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.103542 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e692ac-2613-4916-9879-c60050804b5e None admin] RBAC: Adding query filter params (name=image) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:57.103854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e692ac-2613-4916-9879-c60050804b5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:57.105265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e692ac-2613-4916-9879-c60050804b5e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:57.110873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 525/1006] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:57.116643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a561a29-686b-431d-b5fa-4e60eadda3ff None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:57.179401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:57.240565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a561a29-686b-431d-b5fa-4e60eadda3ff None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:57.288231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da3616-bb4e-4b7e-b5d4-7bfc18a67b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:57.288946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da3616-bb4e-4b7e-b5d4-7bfc18a67b52 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.289235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da3616-bb4e-4b7e-b5d4-7bfc18a67b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.289517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da3616-bb4e-4b7e-b5d4-7bfc18a67b52 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.290238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da3616-bb4e-4b7e-b5d4-7bfc18a67b52 None admin] RBAC: Adding query filter params (type=image) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:57.290548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da3616-bb4e-4b7e-b5d4-7bfc18a67b52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:57.291977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da3616-bb4e-4b7e-b5d4-7bfc18a67b52 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:57.304697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 482/1007] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:57.826686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372c26-8fdd-4cdd-8ab5-5705bddd6d69 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.827045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372c26-8fdd-4cdd-8ab5-5705bddd6d69 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.827255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df372c26-8fdd-4cdd-8ab5-5705bddd6d69 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.828828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 526/1008] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:57.834025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2a12f-24ae-46fd-951f-0a299bc7b359 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:57.834322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2a12f-24ae-46fd-951f-0a299bc7b359 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:57.834630 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c2a12f-24ae-46fd-951f-0a299bc7b359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:57.851977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a0c2a12f-24ae-46fd-951f-0a299bc7b359 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:57.853226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a0c2a12f-24ae-46fd-951f-0a299bc7b359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:57.925147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 483/1009] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:57 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:57.948403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df372c26-8fdd-4cdd-8ab5-5705bddd6d69 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:58.049883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:58.058969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df372c26-8fdd-4cdd-8ab5-5705bddd6d69 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:58.094991 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02157e19-b326-4957-b2a4-15dc14fdaefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:58.095790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02157e19-b326-4957-b2a4-15dc14fdaefd None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:58.096074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02157e19-b326-4957-b2a4-15dc14fdaefd None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:58.096347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02157e19-b326-4957-b2a4-15dc14fdaefd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:58.097616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02157e19-b326-4957-b2a4-15dc14fdaefd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:58.099040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02157e19-b326-4957-b2a4-15dc14fdaefd None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:58.105651 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 527/1010] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:42:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:58.106853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2a12f-24ae-46fd-951f-0a299bc7b359 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:58.151590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:58.210749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c2a12f-24ae-46fd-951f-0a299bc7b359 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:58.249060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d86c68b-a5cd-4370-a2e1-41bf37238793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:58.249631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86c68b-a5cd-4370-a2e1-41bf37238793 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:58.249919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86c68b-a5cd-4370-a2e1-41bf37238793 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:58.250202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86c68b-a5cd-4370-a2e1-41bf37238793 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:58.250910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86c68b-a5cd-4370-a2e1-41bf37238793 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:58.252350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d86c68b-a5cd-4370-a2e1-41bf37238793 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:58.284571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 484/1011] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:42:58 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:42:59.070265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf3a97-d5e3-44a4-9489-54aef5d7320e None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.070819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf3a97-d5e3-44a4-9489-54aef5d7320e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.070819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf3a97-d5e3-44a4-9489-54aef5d7320e None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.072337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 528/1012] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:42:59.077875 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737ec70-44ce-4626-8edb-bf4060540455 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.078165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737ec70-44ce-4626-8edb-bf4060540455 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.078469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2737ec70-44ce-4626-8edb-bf4060540455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.095342 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2737ec70-44ce-4626-8edb-bf4060540455 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:59.096529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2737ec70-44ce-4626-8edb-bf4060540455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:59.163523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 485/1013] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:42:59.186215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf3a97-d5e3-44a4-9489-54aef5d7320e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:59.290612 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf3a97-d5e3-44a4-9489-54aef5d7320e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:59.325696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f3f1c-b6d3-4b3d-a1a5-ef0c09f34c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:59.327020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f3f1c-b6d3-4b3d-a1a5-ef0c09f34c91 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.327304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f3f1c-b6d3-4b3d-a1a5-ef0c09f34c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.327577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f3f1c-b6d3-4b3d-a1a5-ef0c09f34c91 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.328225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f3f1c-b6d3-4b3d-a1a5-ef0c09f34c91 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:59.329650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f3f1c-b6d3-4b3d-a1a5-ef0c09f34c91 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-6b4f3f1c-b6d3-4b3d-a1a5-ef0c09f34c91 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:42:59.341554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:42:59.343051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 11:42:59.344875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:42:59.350908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 529/1014] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:42:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:42:59.351124 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737ec70-44ce-4626-8edb-bf4060540455 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:59.375514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:59.399074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e27aa-f7ae-4e25-baaf-0ec75b381c3c None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.399365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e27aa-f7ae-4e25-baaf-0ec75b381c3c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.399651 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e27aa-f7ae-4e25-baaf-0ec75b381c3c None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.401081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 530/1015] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:42:59.406417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc47ef-a6de-41be-b237-5ed0858770c9 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.406716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc47ef-a6de-41be-b237-5ed0858770c9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.407001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc47ef-a6de-41be-b237-5ed0858770c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.425992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-15cc47ef-a6de-41be-b237-5ed0858770c9 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:42:59.427397 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-15cc47ef-a6de-41be-b237-5ed0858770c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:42:59.453041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:59.467485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2737ec70-44ce-4626-8edb-bf4060540455 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:59.502016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 531/1016] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:42:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:42:59.510357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7224be1-e6e1-4958-93c2-68a3f335155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:59.511078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7224be1-e6e1-4958-93c2-68a3f335155d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.511361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7224be1-e6e1-4958-93c2-68a3f335155d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.511636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7224be1-e6e1-4958-93c2-68a3f335155d None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.512384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7224be1-e6e1-4958-93c2-68a3f335155d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:59.512708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7224be1-e6e1-4958-93c2-68a3f335155d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:59.514624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7224be1-e6e1-4958-93c2-68a3f335155d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:59.519821 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 486/1017] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:59.525225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc47ef-a6de-41be-b237-5ed0858770c9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:59.526025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7224be1-e6e1-4958-93c2-68a3f335155d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:59.623260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:59.635971 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc47ef-a6de-41be-b237-5ed0858770c9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:59.636171 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7224be1-e6e1-4958-93c2-68a3f335155d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:59.673283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d660f-5afc-4479-b791-1764e6b5e4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:59.673965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d660f-5afc-4479-b791-1764e6b5e4f6 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.674249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d660f-5afc-4479-b791-1764e6b5e4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.674535 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d660f-5afc-4479-b791-1764e6b5e4f6 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.675299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d660f-5afc-4479-b791-1764e6b5e4f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:59.676708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7d660f-5afc-4479-b791-1764e6b5e4f6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:59.685178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde2c98-61e1-46ae-b42e-7a530e7104e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:59.686096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde2c98-61e1-46ae-b42e-7a530e7104e9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.686614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde2c98-61e1-46ae-b42e-7a530e7104e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.686898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde2c98-61e1-46ae-b42e-7a530e7104e9 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.687685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde2c98-61e1-46ae-b42e-7a530e7104e9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:42:59.688040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde2c98-61e1-46ae-b42e-7a530e7104e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:59.689472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde2c98-61e1-46ae-b42e-7a530e7104e9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:59.695824 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 532/1018] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 23 11:42:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:42:59.701877 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 487/1019] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:42:59 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:42:59.702133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde2c98-61e1-46ae-b42e-7a530e7104e9 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:42:59.739086 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:42:59.834428 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde2c98-61e1-46ae-b42e-7a530e7104e9 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:42:59.882912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d7cc52-657e-4208-85e1-2ba2452531e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:42:59.883499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7cc52-657e-4208-85e1-2ba2452531e3 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:42:59.883785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7cc52-657e-4208-85e1-2ba2452531e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:42:59.884067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d7cc52-657e-4208-85e1-2ba2452531e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:42:59.884843 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7cc52-657e-4208-85e1-2ba2452531e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:42:59.886214 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d7cc52-657e-4208-85e1-2ba2452531e3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:42:59.909523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 533/1020] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 23 11:42:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:43:00.366153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fe138-b6c6-42cc-aab2-f473e351b79f None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:00.366720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fe138-b6c6-42cc-aab2-f473e351b79f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:00.366801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fe138-b6c6-42cc-aab2-f473e351b79f None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:00.368201 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 488/1021] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:00.373618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e728d-923c-4d96-a7cb-c6ad295a769d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:00.374050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e728d-923c-4d96-a7cb-c6ad295a769d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:00.374246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e728d-923c-4d96-a7cb-c6ad295a769d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:00.391377 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-553e728d-923c-4d96-a7cb-c6ad295a769d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:00.392568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-553e728d-923c-4d96-a7cb-c6ad295a769d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:00.462299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:00.471140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 534/1022] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:00.496953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fe138-b6c6-42cc-aab2-f473e351b79f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:00.604201 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fe138-b6c6-42cc-aab2-f473e351b79f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:00.637788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74976d39-90b7-44ef-8dab-fe2dc06c0c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:00.638555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74976d39-90b7-44ef-8dab-fe2dc06c0c61 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:00.639408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74976d39-90b7-44ef-8dab-fe2dc06c0c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:00.639685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74976d39-90b7-44ef-8dab-fe2dc06c0c61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:00.640939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74976d39-90b7-44ef-8dab-fe2dc06c0c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:00.642360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74976d39-90b7-44ef-8dab-fe2dc06c0c61 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:00.648698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 489/1023] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:00.649927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e728d-923c-4d96-a7cb-c6ad295a769d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:00.761471 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e728d-923c-4d96-a7cb-c6ad295a769d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:00.803475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c478a2-6797-43bf-8cb5-83f0440dd898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:00.804205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c478a2-6797-43bf-8cb5-83f0440dd898 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:00.804499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c478a2-6797-43bf-8cb5-83f0440dd898 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:00.804892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c478a2-6797-43bf-8cb5-83f0440dd898 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:00.805639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c478a2-6797-43bf-8cb5-83f0440dd898 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:00.807106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c478a2-6797-43bf-8cb5-83f0440dd898 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:00.829952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 535/1024] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:43:00 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:01.803978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13c365-3035-4c69-aa5c-eaaf836be733 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:01.804491 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13c365-3035-4c69-aa5c-eaaf836be733 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:01.804570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13c365-3035-4c69-aa5c-eaaf836be733 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:01.806101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 490/1025] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:01.811683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdda807-4b67-452e-9319-e297a11f0cf7 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:01.812122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdda807-4b67-452e-9319-e297a11f0cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:01.812270 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdda807-4b67-452e-9319-e297a11f0cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:01.829058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3bdda807-4b67-452e-9319-e297a11f0cf7 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:01.830272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3bdda807-4b67-452e-9319-e297a11f0cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:01.883368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:01.902409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 536/1026] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:01.925978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13c365-3035-4c69-aa5c-eaaf836be733 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:01.945852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:02.035535 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13c365-3035-4c69-aa5c-eaaf836be733 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.070774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330811cd-2502-4fb3-a420-d06ccaa08241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.071579 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330811cd-2502-4fb3-a420-d06ccaa08241 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.071914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330811cd-2502-4fb3-a420-d06ccaa08241 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.072139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330811cd-2502-4fb3-a420-d06ccaa08241 None admin] PATH_INFO: `/v3/services/image` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.072846 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330811cd-2502-4fb3-a420-d06ccaa08241 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.074236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330811cd-2502-4fb3-a420-d06ccaa08241 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.076948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898aca8-4ff7-4d30-bb07-c56806698746 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.077238 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898aca8-4ff7-4d30-bb07-c56806698746 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.077513 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898aca8-4ff7-4d30-bb07-c56806698746 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.079123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 537/1027] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:02.084026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad626fb5-2980-412e-8430-d265d5a9c3e9 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.084321 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad626fb5-2980-412e-8430-d265d5a9c3e9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.084607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad626fb5-2980-412e-8430-d265d5a9c3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-330811cd-2502-4fb3-a420-d06ccaa08241 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:02.087391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:02.088482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:43:02.089374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:02.093851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 491/1028] 10.222.0.67 () {56 vars in 1105 bytes} [Thu Jun 23 11:43:01 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:02.094895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330811cd-2502-4fb3-a420-d06ccaa08241 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.106410 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ad626fb5-2980-412e-8430-d265d5a9c3e9 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:02.107949 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ad626fb5-2980-412e-8430-d265d5a9c3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:02.178198 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:02.182989 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 538/1029] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:43:02.202861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330811cd-2502-4fb3-a420-d06ccaa08241 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.207062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad626fb5-2980-412e-8430-d265d5a9c3e9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.239148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd7318-93cf-41f5-82bc-f7554592eb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.239815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd7318-93cf-41f5-82bc-f7554592eb6d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.240103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd7318-93cf-41f5-82bc-f7554592eb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.240385 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fd7318-93cf-41f5-82bc-f7554592eb6d None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.241125 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd7318-93cf-41f5-82bc-f7554592eb6d None admin] RBAC: Adding query filter params (name=image) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:02.241438 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd7318-93cf-41f5-82bc-f7554592eb6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.242805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fd7318-93cf-41f5-82bc-f7554592eb6d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.248461 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 492/1030] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:02.255123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd7318-93cf-41f5-82bc-f7554592eb6d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.326870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad626fb5-2980-412e-8430-d265d5a9c3e9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.345177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:02.361313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fd7318-93cf-41f5-82bc-f7554592eb6d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.367212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bee28-3c88-48e9-81fe-4d5cec709499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.368153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bee28-3c88-48e9-81fe-4d5cec709499 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.368459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bee28-3c88-48e9-81fe-4d5cec709499 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.368778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85bee28-3c88-48e9-81fe-4d5cec709499 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.380285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85bee28-3c88-48e9-81fe-4d5cec709499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-f85bee28-3c88-48e9-81fe-4d5cec709499 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:43:02.389208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:02.390192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:43:02.391083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 539/1031] 10.222.0.67 () {56 vars in 1120 bytes} [Thu Jun 23 11:43:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:43:02.396376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bee28-3c88-48e9-81fe-4d5cec709499 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.400619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ae8bf-e9ab-45ce-91e7-ea88a9cdded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.401330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ae8bf-e9ab-45ce-91e7-ea88a9cdded1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.401620 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ae8bf-e9ab-45ce-91e7-ea88a9cdded1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.401903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ae8bf-e9ab-45ce-91e7-ea88a9cdded1 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.402612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ae8bf-e9ab-45ce-91e7-ea88a9cdded1 None admin] RBAC: Adding query filter params (type=image) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:02.402923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ae8bf-e9ab-45ce-91e7-ea88a9cdded1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.404319 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530ae8bf-e9ab-45ce-91e7-ea88a9cdded1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.415590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 493/1032] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:02.416646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ae8bf-e9ab-45ce-91e7-ea88a9cdded1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.428712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:02.508020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85bee28-3c88-48e9-81fe-4d5cec709499 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.538745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ae8bf-e9ab-45ce-91e7-ea88a9cdded1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.552804 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd437a-9ae7-473e-aad2-140aa6a5df53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.553479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd437a-9ae7-473e-aad2-140aa6a5df53 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.553773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd437a-9ae7-473e-aad2-140aa6a5df53 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.554050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd437a-9ae7-473e-aad2-140aa6a5df53 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.554806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd437a-9ae7-473e-aad2-140aa6a5df53 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:02.555122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd437a-9ae7-473e-aad2-140aa6a5df53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.556492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd437a-9ae7-473e-aad2-140aa6a5df53 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.566149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 540/1033] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 23 11:43:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:02.572273 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd437a-9ae7-473e-aad2-140aa6a5df53 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.581360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a8cd1-83f8-4570-a51d-dc30c3dbe6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.582325 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a8cd1-83f8-4570-a51d-dc30c3dbe6ea None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.582799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a8cd1-83f8-4570-a51d-dc30c3dbe6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.583281 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708a8cd1-83f8-4570-a51d-dc30c3dbe6ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.584311 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a8cd1-83f8-4570-a51d-dc30c3dbe6ea None admin] RBAC: Adding query filter params (interface=public, service_id=0fe41a516ce147908673c7da59031643, region_id=RegionOne) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:02.584927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a8cd1-83f8-4570-a51d-dc30c3dbe6ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.586528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708a8cd1-83f8-4570-a51d-dc30c3dbe6ea None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.592022 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 494/1034] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 23 11:43:02 2022] GET /identity/v3/endpoints?service_id=0fe41a516ce147908673c7da59031643&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:02.597490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a8cd1-83f8-4570-a51d-dc30c3dbe6ea None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.617895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:02.680310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:02.693583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd437a-9ae7-473e-aad2-140aa6a5df53 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.713039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708a8cd1-83f8-4570-a51d-dc30c3dbe6ea None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.728910 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e1c74-99b2-42f1-b919-55cd0dfc51d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.729706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e1c74-99b2-42f1-b919-55cd0dfc51d4 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.729993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e1c74-99b2-42f1-b919-55cd0dfc51d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.730273 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e1c74-99b2-42f1-b919-55cd0dfc51d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.741010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e1c74-99b2-42f1-b919-55cd0dfc51d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.742476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e1c74-99b2-42f1-b919-55cd0dfc51d4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.749474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730140d-aef8-4d6c-85ea-4b4552765c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.750526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730140d-aef8-4d6c-85ea-4b4552765c63 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.750959 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730140d-aef8-4d6c-85ea-4b4552765c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.751384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730140d-aef8-4d6c-85ea-4b4552765c63 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.752202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730140d-aef8-4d6c-85ea-4b4552765c63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.753745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730140d-aef8-4d6c-85ea-4b4552765c63 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8f2e1c74-99b2-42f1-b919-55cd0dfc51d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:43:02.759475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:02.760537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:02.761669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:43:02.761669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:43:02.761669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:02.761669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:43:02.772717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 541/1035] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:43:02.772717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e1c74-99b2-42f1-b919-55cd0dfc51d4 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:02.773041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 495/1036] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 23 11:43:02 2022] GET /identity/v3/services => generated 1957 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:43:02.881661 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:02.894645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e1c74-99b2-42f1-b919-55cd0dfc51d4 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:02.923608 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad1c5b-daf9-4694-85bb-5e25c3ab3bc1 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.924302 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad1c5b-daf9-4694-85bb-5e25c3ab3bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.924302 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad1c5b-daf9-4694-85bb-5e25c3ab3bc1 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.925643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 496/1037] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:43:02.931039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e447f-988d-4123-a21f-f914dbc76ede None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.931259 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e447f-988d-4123-a21f-f914dbc76ede None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.931536 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e447f-988d-4123-a21f-f914dbc76ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.947745 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295e551-309f-49c6-8b57-5c88e2b3d39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:02.948428 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295e551-309f-49c6-8b57-5c88e2b3d39d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:02.948751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295e551-309f-49c6-8b57-5c88e2b3d39d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:02.949041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295e551-309f-49c6-8b57-5c88e2b3d39d None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:02.949776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295e551-309f-49c6-8b57-5c88e2b3d39d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:02.950099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295e551-309f-49c6-8b57-5c88e2b3d39d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:02.952211 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ac0e447f-988d-4123-a21f-f914dbc76ede None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:02.952451 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a295e551-309f-49c6-8b57-5c88e2b3d39d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:02.953529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ac0e447f-988d-4123-a21f-f914dbc76ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:02.978355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 542/1038] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:02.979884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295e551-309f-49c6-8b57-5c88e2b3d39d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.031358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 497/1039] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:43:03.055431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e447f-988d-4123-a21f-f914dbc76ede None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.100148 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295e551-309f-49c6-8b57-5c88e2b3d39d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.121823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:03.138182 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9734c-f541-449d-afa3-e02e5b8221fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.139548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9734c-f541-449d-afa3-e02e5b8221fb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.139832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9734c-f541-449d-afa3-e02e5b8221fb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.140106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9734c-f541-449d-afa3-e02e5b8221fb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.147575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9734c-f541-449d-afa3-e02e5b8221fb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.149040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9734c-f541-449d-afa3-e02e5b8221fb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.159254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e447f-988d-4123-a21f-f914dbc76ede None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-1fb9734c-f541-449d-afa3-e02e5b8221fb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:43:03.163976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:03.165041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:03.166049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:03.166049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:03.166049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:43:03.166049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:43:03.166049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:43:03.166049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:43:03.166049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:43:03.166049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:43:03.166049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:43:03.166049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:43:03.168998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 543/1040] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 23 11:43:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:43:03.170859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9734c-f541-449d-afa3-e02e5b8221fb None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.183082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:03.199386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8829bff-3ca9-4cf0-b730-9877a3d4bda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.200185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8829bff-3ca9-4cf0-b730-9877a3d4bda8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.200484 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8829bff-3ca9-4cf0-b730-9877a3d4bda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.200767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8829bff-3ca9-4cf0-b730-9877a3d4bda8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.211831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8829bff-3ca9-4cf0-b730-9877a3d4bda8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d8829bff-3ca9-4cf0-b730-9877a3d4bda8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:43:03.220991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:03.222993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:03.223879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 498/1041] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:43:03.227386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8829bff-3ca9-4cf0-b730-9877a3d4bda8 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.294242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9734c-f541-449d-afa3-e02e5b8221fb None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.329319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46408906-020e-466b-9277-18c7154c0675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.329986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46408906-020e-466b-9277-18c7154c0675 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.330275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46408906-020e-466b-9277-18c7154c0675 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.330554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46408906-020e-466b-9277-18c7154c0675 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.331383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46408906-020e-466b-9277-18c7154c0675 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:03.331695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46408906-020e-466b-9277-18c7154c0675 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.333133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46408906-020e-466b-9277-18c7154c0675 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.333967 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8829bff-3ca9-4cf0-b730-9877a3d4bda8 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.351394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 544/1042] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:03.356897 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46408906-020e-466b-9277-18c7154c0675 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.360477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:03.371712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ffc37-c13c-41d5-b712-f57d384a8b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.372382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ffc37-c13c-41d5-b712-f57d384a8b60 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.372666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ffc37-c13c-41d5-b712-f57d384a8b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.373015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69ffc37-c13c-41d5-b712-f57d384a8b60 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.373763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ffc37-c13c-41d5-b712-f57d384a8b60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:03.374069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ffc37-c13c-41d5-b712-f57d384a8b60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.375430 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69ffc37-c13c-41d5-b712-f57d384a8b60 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.384220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 499/1043] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:03.390380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ffc37-c13c-41d5-b712-f57d384a8b60 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.427854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:03.466232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46408906-020e-466b-9277-18c7154c0675 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.497363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69ffc37-c13c-41d5-b712-f57d384a8b60 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.501849 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d29a8-f26c-4ba4-b590-e8970c6ebd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.502501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d29a8-f26c-4ba4-b590-e8970c6ebd2c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.502784 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d29a8-f26c-4ba4-b590-e8970c6ebd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.503059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d29a8-f26c-4ba4-b590-e8970c6ebd2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.534099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35d8a4-7b62-40b1-aeda-142846fd2982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.534963 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35d8a4-7b62-40b1-aeda-142846fd2982 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.535246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35d8a4-7b62-40b1-aeda-142846fd2982 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.535519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35d8a4-7b62-40b1-aeda-142846fd2982 None admin] PATH_INFO: `/v3/users/68da1e7aa7714d0488d1a8bf3245944b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.550326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35d8a4-7b62-40b1-aeda-142846fd2982 None admin] RBAC: Authorizing `identity:get_user(user_id=68da1e7aa7714d0488d1a8bf3245944b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.551793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35d8a4-7b62-40b1-aeda-142846fd2982 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.553808 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 500/1044] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/users/68da1e7aa7714d0488d1a8bf3245944b => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:03.559158 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35d8a4-7b62-40b1-aeda-142846fd2982 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.578739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d29a8-f26c-4ba4-b590-e8970c6ebd2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.580207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d29a8-f26c-4ba4-b590-e8970c6ebd2c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.610397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:03.643590 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:03.678315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35d8a4-7b62-40b1-aeda-142846fd2982 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.680360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 545/1045] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 11:43:03.686485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d29a8-f26c-4ba4-b590-e8970c6ebd2c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.715518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9ba6d-4d63-40af-a495-04a82fae92d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.716177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9ba6d-4d63-40af-a495-04a82fae92d3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.716523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9ba6d-4d63-40af-a495-04a82fae92d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.716849 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9ba6d-4d63-40af-a495-04a82fae92d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.786842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9ba6d-4d63-40af-a495-04a82fae92d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.788295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9ba6d-4d63-40af-a495-04a82fae92d3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.789347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d29a8-f26c-4ba4-b590-e8970c6ebd2c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:03.826818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb08485-04ac-4e58-bb05-8f49f7d26d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:03.828011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb08485-04ac-4e58-bb05-8f49f7d26d58 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:03.828496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb08485-04ac-4e58-bb05-8f49f7d26d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:03.829012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb08485-04ac-4e58-bb05-8f49f7d26d58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:03.837016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:03.840120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb08485-04ac-4e58-bb05-8f49f7d26d58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:03.841831 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb08485-04ac-4e58-bb05-8f49f7d26d58 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-4cb08485-04ac-4e58-bb05-8f49f7d26d58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:43:03.861472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:03.862439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:03.863553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:43:03.863553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:43:03.863553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:03.863553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:43:03.863553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 546/1046] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:43:03.868378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb08485-04ac-4e58-bb05-8f49f7d26d58 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.892531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:03.896912 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 501/1047] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 11:43:03.903058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9ba6d-4d63-40af-a495-04a82fae92d3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:03.986321 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb08485-04ac-4e58-bb05-8f49f7d26d58 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:04.005448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9ba6d-4d63-40af-a495-04a82fae92d3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:04.023287 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3f132-473a-4e1a-a364-b53166e90371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:04.024191 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3f132-473a-4e1a-a364-b53166e90371 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:04.024641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3f132-473a-4e1a-a364-b53166e90371 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:04.025231 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a3f132-473a-4e1a-a364-b53166e90371 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:04.025962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3f132-473a-4e1a-a364-b53166e90371 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:04.026273 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3f132-473a-4e1a-a364-b53166e90371 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:04.027670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a3f132-473a-4e1a-a364-b53166e90371 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:04.039273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaa886-3c8b-4757-a096-d78f3edd9653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:04.040086 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 547/1048] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:04.040199 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaa886-3c8b-4757-a096-d78f3edd9653 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:04.040482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaa886-3c8b-4757-a096-d78f3edd9653 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:04.040790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaa886-3c8b-4757-a096-d78f3edd9653 None admin] PATH_INFO: `/v3/projects/9f97e0b2639b489098f7163c943324e8` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:04.042060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaa886-3c8b-4757-a096-d78f3edd9653 None admin] RBAC: Authorizing `identity:get_project(project_id=9f97e0b2639b489098f7163c943324e8)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:04.043483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaa886-3c8b-4757-a096-d78f3edd9653 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:04.050728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 502/1049] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:43:03 2022] GET /identity/v3/projects/9f97e0b2639b489098f7163c943324e8 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:04.050891 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3f132-473a-4e1a-a364-b53166e90371 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:04.051610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaa886-3c8b-4757-a096-d78f3edd9653 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:04.069790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:04.137958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:04.160092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaa886-3c8b-4757-a096-d78f3edd9653 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:04.160725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a3f132-473a-4e1a-a364-b53166e90371 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:04.196180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff46b9-320c-4f95-b319-18a9d5509d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:04.196995 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff46b9-320c-4f95-b319-18a9d5509d5e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:04.197280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff46b9-320c-4f95-b319-18a9d5509d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:04.197557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff46b9-320c-4f95-b319-18a9d5509d5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:04.198436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaeb11-b4c1-4ebf-aa15-5ff4b76cea8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:04.199087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaeb11-b4c1-4ebf-aa15-5ff4b76cea8f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:04.199486 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaeb11-b4c1-4ebf-aa15-5ff4b76cea8f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:04.199766 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddaeb11-b4c1-4ebf-aa15-5ff4b76cea8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:04.200629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddaeb11-b4c1-4ebf-aa15-5ff4b76cea8f None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=9f97e0b2639b489098f7163c943324e8, user.id=68da1e7aa7714d0488d1a8bf3245944b) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:04.201006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddaeb11-b4c1-4ebf-aa15-5ff4b76cea8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:04.202442 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddaeb11-b4c1-4ebf-aa15-5ff4b76cea8f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:04.204320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff46b9-320c-4f95-b319-18a9d5509d5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:04.205784 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff46b9-320c-4f95-b319-18a9d5509d5e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:04.208332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 503/1050] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:43:04 2022] GET /identity/v3/role_assignments?user.id=68da1e7aa7714d0488d1a8bf3245944b&scope.project.id=9f97e0b2639b489098f7163c943324e8&role.id=569707378794437b93d168fc8bb9cbc3 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-48ff46b9-320c-4f95-b319-18a9d5509d5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:43:04.221177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:04.222170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:04.223640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:43:04.223640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:43:04.223640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:43:04.223640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:43:04.223640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:43:04.223640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:43:04.223640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:43:04.223640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:43:04.223640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 548/1051] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:43:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:43:04.229093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaeb11-b4c1-4ebf-aa15-5ff4b76cea8f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:04.325643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:04.351131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddaeb11-b4c1-4ebf-aa15-5ff4b76cea8f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:04.390161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc178561-e63a-474d-8b97-83528d32a4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:04.390826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178561-e63a-474d-8b97-83528d32a4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:04.391116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178561-e63a-474d-8b97-83528d32a4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:04.391394 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178561-e63a-474d-8b97-83528d32a4b3 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:04.392185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc178561-e63a-474d-8b97-83528d32a4b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:04.392549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc178561-e63a-474d-8b97-83528d32a4b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:04.394039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc178561-e63a-474d-8b97-83528d32a4b3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:04.411528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 504/1052] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:43:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:04.413149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff46b9-320c-4f95-b319-18a9d5509d5e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:04.517900 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff46b9-320c-4f95-b319-18a9d5509d5e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:04.555503 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4873e23-10c0-4fd4-aa97-dc45cfb03a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:04.556171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4873e23-10c0-4fd4-aa97-dc45cfb03a2b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:04.556463 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4873e23-10c0-4fd4-aa97-dc45cfb03a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:04.556766 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4873e23-10c0-4fd4-aa97-dc45cfb03a2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:04.557609 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4873e23-10c0-4fd4-aa97-dc45cfb03a2b None admin] RBAC: Adding query filter params (role.id=370db8aebcaf44e995d905fdd73e0bbd, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=36c18bbd22f74efbae7aab1cdd3f6403) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:04.557925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4873e23-10c0-4fd4-aa97-dc45cfb03a2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:04.559358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4873e23-10c0-4fd4-aa97-dc45cfb03a2b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:04.572703 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 549/1053] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:43:04 2022] GET /identity/v3/role_assignments?user.id=36c18bbd22f74efbae7aab1cdd3f6403&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=370db8aebcaf44e995d905fdd73e0bbd => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:04.873220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56c28a-e7c4-4ab1-b757-cf15085e1aed None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:04.873550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56c28a-e7c4-4ab1-b757-cf15085e1aed None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:04.873801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56c28a-e7c4-4ab1-b757-cf15085e1aed None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:04.875356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 505/1054] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:04.880430 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b1468-0702-4d4d-99e8-a2481b5ae5fe None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:04.880758 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b1468-0702-4d4d-99e8-a2481b5ae5fe None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:04.881046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b1468-0702-4d4d-99e8-a2481b5ae5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:04.887063 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:04.899699 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-097b1468-0702-4d4d-99e8-a2481b5ae5fe None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:04.900963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-097b1468-0702-4d4d-99e8-a2481b5ae5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:04.972698 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 550/1055] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:43:05.000600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56c28a-e7c4-4ab1-b757-cf15085e1aed None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:05.111556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56c28a-e7c4-4ab1-b757-cf15085e1aed None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:05.145316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0452d8-7157-4ff1-9e17-834f7fe550b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:05.146103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0452d8-7157-4ff1-9e17-834f7fe550b2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:05.146386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0452d8-7157-4ff1-9e17-834f7fe550b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:05.146665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0452d8-7157-4ff1-9e17-834f7fe550b2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:05.147316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0452d8-7157-4ff1-9e17-834f7fe550b2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:05.148710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0452d8-7157-4ff1-9e17-834f7fe550b2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0b0452d8-7157-4ff1-9e17-834f7fe550b2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:05.160427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:05.161618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 11:43:05.162488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:05.165000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 506/1056] 10.222.0.67 () {56 vars in 1105 bytes} [Thu Jun 23 11:43:04 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:05.167725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b1468-0702-4d4d-99e8-a2481b5ae5fe None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:05.270910 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b1468-0702-4d4d-99e8-a2481b5ae5fe None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:05.306483 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31dfc1c-df2b-4395-be90-1854c938ac73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:05.307185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31dfc1c-df2b-4395-be90-1854c938ac73 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:05.307476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31dfc1c-df2b-4395-be90-1854c938ac73 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:05.307760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31dfc1c-df2b-4395-be90-1854c938ac73 None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:05.308502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31dfc1c-df2b-4395-be90-1854c938ac73 None admin] RBAC: Adding query filter params (name=image) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:05.308857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31dfc1c-df2b-4395-be90-1854c938ac73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:05.310815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31dfc1c-df2b-4395-be90-1854c938ac73 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:05.312168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:05.322262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 551/1057] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:05.323987 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0452d8-7157-4ff1-9e17-834f7fe550b2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:05.327506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:05.426236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0452d8-7157-4ff1-9e17-834f7fe550b2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:05.459965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aead83-12f7-4ede-ac73-16c3b49fc0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:05.460641 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aead83-12f7-4ede-ac73-16c3b49fc0ae None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:05.460964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aead83-12f7-4ede-ac73-16c3b49fc0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:05.461241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aead83-12f7-4ede-ac73-16c3b49fc0ae None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:05.461954 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aead83-12f7-4ede-ac73-16c3b49fc0ae None admin] RBAC: Adding query filter params (type=image) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:05.462269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aead83-12f7-4ede-ac73-16c3b49fc0ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:05.463634 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aead83-12f7-4ede-ac73-16c3b49fc0ae None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:05.473180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 507/1058] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:05 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:05.474686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31dfc1c-df2b-4395-be90-1854c938ac73 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:05.581623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31dfc1c-df2b-4395-be90-1854c938ac73 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:05.616599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa7d4d-1f62-4b15-8074-80f37eb77837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:05.617399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa7d4d-1f62-4b15-8074-80f37eb77837 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:05.617751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa7d4d-1f62-4b15-8074-80f37eb77837 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:05.618104 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa7d4d-1f62-4b15-8074-80f37eb77837 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:05.618909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa7d4d-1f62-4b15-8074-80f37eb77837 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:05.620355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa7d4d-1f62-4b15-8074-80f37eb77837 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:05.646197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 552/1059] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 23 11:43:05 2022] POST /identity/v3/endpoints => generated 327 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:06.235265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a86eb-32c9-45ca-b781-814dc79e2f65 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:06.235792 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a86eb-32c9-45ca-b781-814dc79e2f65 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:06.235792 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a86eb-32c9-45ca-b781-814dc79e2f65 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:06.237394 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 508/1060] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:06.242617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3006c-f08d-41a6-a2a2-7f095d837479 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:06.243087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3006c-f08d-41a6-a2a2-7f095d837479 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:06.243165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3006c-f08d-41a6-a2a2-7f095d837479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:06.260150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e6a3006c-f08d-41a6-a2a2-7f095d837479 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:06.261362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e6a3006c-f08d-41a6-a2a2-7f095d837479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:06.312030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:06.338685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 553/1061] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:06.362358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a86eb-32c9-45ca-b781-814dc79e2f65 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:06.435218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:06.473234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a86eb-32c9-45ca-b781-814dc79e2f65 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:06.508336 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13863b6f-c627-400b-b753-0e9c75bf72dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:06.509071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13863b6f-c627-400b-b753-0e9c75bf72dd None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:06.509361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13863b6f-c627-400b-b753-0e9c75bf72dd None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:06.509639 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13863b6f-c627-400b-b753-0e9c75bf72dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:06.521000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13863b6f-c627-400b-b753-0e9c75bf72dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-13863b6f-c627-400b-b753-0e9c75bf72dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:43:06.528866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:06.530507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:06.531360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:06.531360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:06.531360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:06.531360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:06.531360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:43:06.531360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:43:06.531360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:43:06.531360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:43:06.531360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:43:06.531360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:43:06.531360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:06.531360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:06.532830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 509/1062] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:43:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:06.537186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3006c-f08d-41a6-a2a2-7f095d837479 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:06.627756 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa4aa2-0e81-4787-9461-f5601c5ea057 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:06.628082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa4aa2-0e81-4787-9461-f5601c5ea057 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:06.628332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa4aa2-0e81-4787-9461-f5601c5ea057 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:06.629913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 510/1063] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:06.634634 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f5793-64ed-4d4e-890c-47108232f91b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:06.634929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f5793-64ed-4d4e-890c-47108232f91b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:06.635218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86f5793-64ed-4d4e-890c-47108232f91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:06.643409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3006c-f08d-41a6-a2a2-7f095d837479 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:06.652145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a86f5793-64ed-4d4e-890c-47108232f91b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:06.653379 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a86f5793-64ed-4d4e-890c-47108232f91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:06.691262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e6191-1cc6-4e29-993d-b8a9be1fd435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:06.692614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e6191-1cc6-4e29-993d-b8a9be1fd435 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:06.692614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e6191-1cc6-4e29-993d-b8a9be1fd435 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:06.692949 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e6191-1cc6-4e29-993d-b8a9be1fd435 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:06.694103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e6191-1cc6-4e29-993d-b8a9be1fd435 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:06.694534 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e6191-1cc6-4e29-993d-b8a9be1fd435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:06.696995 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e6191-1cc6-4e29-993d-b8a9be1fd435 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:06.710269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 554/1064] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:43:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:06.717542 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e6191-1cc6-4e29-993d-b8a9be1fd435 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:06.745168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 511/1065] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:43:06.765603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f5793-64ed-4d4e-890c-47108232f91b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:06.767638 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:06.827952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:06.836298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e6191-1cc6-4e29-993d-b8a9be1fd435 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:06.874350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86f5793-64ed-4d4e-890c-47108232f91b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:06.877497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f4f8f-c026-47a5-9a45-9ddd08f97f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:06.879031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f4f8f-c026-47a5-9a45-9ddd08f97f8e None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:06.879322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f4f8f-c026-47a5-9a45-9ddd08f97f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:06.879606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f4f8f-c026-47a5-9a45-9ddd08f97f8e None admin] PATH_INFO: `/v3/users/68da1e7aa7714d0488d1a8bf3245944b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:06.880906 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f4f8f-c026-47a5-9a45-9ddd08f97f8e None admin] RBAC: Authorizing `identity:get_user(user_id=68da1e7aa7714d0488d1a8bf3245944b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:06.882529 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f4f8f-c026-47a5-9a45-9ddd08f97f8e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:06.884317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 555/1066] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:06 2022] GET /identity/v3/users/68da1e7aa7714d0488d1a8bf3245944b => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:06.890666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f4f8f-c026-47a5-9a45-9ddd08f97f8e None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:06.911405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c91cc-57c2-4d62-9be5-e4e7ecfe9709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:06.912226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c91cc-57c2-4d62-9be5-e4e7ecfe9709 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:06.912515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c91cc-57c2-4d62-9be5-e4e7ecfe9709 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:06.912871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c91cc-57c2-4d62-9be5-e4e7ecfe9709 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:06.923648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c91cc-57c2-4d62-9be5-e4e7ecfe9709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-698c91cc-57c2-4d62-9be5-e4e7ecfe9709 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:43:06.931985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:06.933343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:06.934226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 512/1067] 10.222.0.67 () {56 vars in 1120 bytes} [Thu Jun 23 11:43:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:43:06.939689 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c91cc-57c2-4d62-9be5-e4e7ecfe9709 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:07.001626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f4f8f-c026-47a5-9a45-9ddd08f97f8e None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:07.014149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:07.041605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37fb53f-e60a-42d7-9d3c-ffb028111d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:07.042485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fb53f-e60a-42d7-9d3c-ffb028111d19 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:07.042898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fb53f-e60a-42d7-9d3c-ffb028111d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:07.043169 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fb53f-e60a-42d7-9d3c-ffb028111d19 None admin] PATH_INFO: `/v3/projects/9f97e0b2639b489098f7163c943324e8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:07.044451 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37fb53f-e60a-42d7-9d3c-ffb028111d19 None admin] RBAC: Authorizing `identity:get_project(project_id=9f97e0b2639b489098f7163c943324e8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:07.046546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37fb53f-e60a-42d7-9d3c-ffb028111d19 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:07.052963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 556/1068] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:43:06 2022] GET /identity/v3/projects/9f97e0b2639b489098f7163c943324e8 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:07.053110 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c91cc-57c2-4d62-9be5-e4e7ecfe9709 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:07.054020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37fb53f-e60a-42d7-9d3c-ffb028111d19 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:07.069603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:07.089259 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a768cb1-bc9a-46d7-8ed6-05722b16bdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:07.089930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a768cb1-bc9a-46d7-8ed6-05722b16bdcf None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:07.090220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a768cb1-bc9a-46d7-8ed6-05722b16bdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:07.090502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a768cb1-bc9a-46d7-8ed6-05722b16bdcf None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:07.091255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a768cb1-bc9a-46d7-8ed6-05722b16bdcf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:07.091597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a768cb1-bc9a-46d7-8ed6-05722b16bdcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:07.092993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a768cb1-bc9a-46d7-8ed6-05722b16bdcf None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:07.105719 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 513/1069] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 23 11:43:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:07.106790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a768cb1-bc9a-46d7-8ed6-05722b16bdcf None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:07.165702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37fb53f-e60a-42d7-9d3c-ffb028111d19 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:07.202685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c1e22-f989-424b-b22b-767264f5aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:07.203589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c1e22-f989-424b-b22b-767264f5aac6 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:07.204305 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c1e22-f989-424b-b22b-767264f5aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:07.204606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c1e22-f989-424b-b22b-767264f5aac6 None admin] PATH_INFO: `/v3/projects/9f97e0b2639b489098f7163c943324e8/users/68da1e7aa7714d0488d1a8bf3245944b/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:07.206666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c1e22-f989-424b-b22b-767264f5aac6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f97e0b2639b489098f7163c943324e8, user_id=68da1e7aa7714d0488d1a8bf3245944b, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:07.208104 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c1e22-f989-424b-b22b-767264f5aac6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:07.218405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a768cb1-bc9a-46d7-8ed6-05722b16bdcf None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:07.222521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 557/1070] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:43:07 2022] PUT /identity/v3/projects/9f97e0b2639b489098f7163c943324e8/users/68da1e7aa7714d0488d1a8bf3245944b/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:07.265795 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce1bfb-6c0a-4559-bb5b-29583361701f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:07.266609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce1bfb-6c0a-4559-bb5b-29583361701f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:07.266898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce1bfb-6c0a-4559-bb5b-29583361701f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:07.267176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce1bfb-6c0a-4559-bb5b-29583361701f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:07.280463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce1bfb-6c0a-4559-bb5b-29583361701f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:07.282111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce1bfb-6c0a-4559-bb5b-29583361701f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d4ce1bfb-6c0a-4559-bb5b-29583361701f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:43:07.301928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:07.302909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:07.303820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:43:07.303820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:43:07.303820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:07.303820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:07.305034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 514/1071] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:07.309084 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c1e22-f989-424b-b22b-767264f5aac6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:07.354872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:07.423112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c1e22-f989-424b-b22b-767264f5aac6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:07.456760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01242316-15cc-4211-9824-f9b4c6d3cf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:07.457426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01242316-15cc-4211-9824-f9b4c6d3cf47 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:07.457710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01242316-15cc-4211-9824-f9b4c6d3cf47 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:07.457990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01242316-15cc-4211-9824-f9b4c6d3cf47 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:07.458712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01242316-15cc-4211-9824-f9b4c6d3cf47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:07.459022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01242316-15cc-4211-9824-f9b4c6d3cf47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:07.460386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01242316-15cc-4211-9824-f9b4c6d3cf47 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:07.472781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 558/1072] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:07.478585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce1bfb-6c0a-4559-bb5b-29583361701f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:07.513956 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:07.592728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce1bfb-6c0a-4559-bb5b-29583361701f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:07.626177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6acb22d-ba38-43f3-b4ab-7e650ee4dc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:07.626980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acb22d-ba38-43f3-b4ab-7e650ee4dc6b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:07.627349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acb22d-ba38-43f3-b4ab-7e650ee4dc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:07.627547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6acb22d-ba38-43f3-b4ab-7e650ee4dc6b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:07.635674 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6acb22d-ba38-43f3-b4ab-7e650ee4dc6b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:07.637181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6acb22d-ba38-43f3-b4ab-7e650ee4dc6b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-e6acb22d-ba38-43f3-b4ab-7e650ee4dc6b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:43:07.651476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:07.652564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:07.654618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:07.654618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:07.654618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:43:07.654618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:43:07.654618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:43:07.654618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:43:07.654618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:43:07.654618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:43:07.654618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:43:07.654618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:07.659133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 515/1073] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 23 11:43:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:07.660406 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01242316-15cc-4211-9824-f9b4c6d3cf47 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:07.767634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:07.771924 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01242316-15cc-4211-9824-f9b4c6d3cf47 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:07.791040 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160d878-1811-4a99-87d6-8b1e5cfc4bd2 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:07.791338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160d878-1811-4a99-87d6-8b1e5cfc4bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:07.791609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160d878-1811-4a99-87d6-8b1e5cfc4bd2 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:07.793023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 516/1074] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:07.798370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180f29a-3366-42f6-8fd4-87d5f9f0aadc None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:07.798657 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180f29a-3366-42f6-8fd4-87d5f9f0aadc None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:07.798939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180f29a-3366-42f6-8fd4-87d5f9f0aadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:07.820815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd7291-b2f8-4687-a38f-d97fde0fdf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:07.821009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6180f29a-3366-42f6-8fd4-87d5f9f0aadc None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:07.821940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd7291-b2f8-4687-a38f-d97fde0fdf4c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:07.822081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6180f29a-3366-42f6-8fd4-87d5f9f0aadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:07.822454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd7291-b2f8-4687-a38f-d97fde0fdf4c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:07.822694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd7291-b2f8-4687-a38f-d97fde0fdf4c None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:07.823538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd7291-b2f8-4687-a38f-d97fde0fdf4c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:07.823853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd7291-b2f8-4687-a38f-d97fde0fdf4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:07.825324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd7291-b2f8-4687-a38f-d97fde0fdf4c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:07.843877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 559/1075] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 23 11:43:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:07.849683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd7291-b2f8-4687-a38f-d97fde0fdf4c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:07.901276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 517/1076] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:43:07.923738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180f29a-3366-42f6-8fd4-87d5f9f0aadc None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:07.945186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:07.960279 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd7291-b2f8-4687-a38f-d97fde0fdf4c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:07.996259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6a8af-bad1-445c-8ca3-56f545a5f80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:07.997458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6a8af-bad1-445c-8ca3-56f545a5f80d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:07.998549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6a8af-bad1-445c-8ca3-56f545a5f80d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:07.998887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6a8af-bad1-445c-8ca3-56f545a5f80d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:08.009584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6a8af-bad1-445c-8ca3-56f545a5f80d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:08.011048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6a8af-bad1-445c-8ca3-56f545a5f80d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:08.027491 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180f29a-3366-42f6-8fd4-87d5f9f0aadc None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e3d6a8af-bad1-445c-8ca3-56f545a5f80d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:43:08.028726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:08.029708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:08.031097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:43:08.031097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:43:08.031097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:08.031097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:43:08.031097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 560/1077] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:43:08.035753 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6a8af-bad1-445c-8ca3-56f545a5f80d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:08.046900 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:08.066627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4f105-c99d-4df7-953f-51a2ac9aa47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:08.067268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f105-c99d-4df7-953f-51a2ac9aa47f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:08.067550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f105-c99d-4df7-953f-51a2ac9aa47f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:08.067825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4f105-c99d-4df7-953f-51a2ac9aa47f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:08.143666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4f105-c99d-4df7-953f-51a2ac9aa47f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:08.145222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4f105-c99d-4df7-953f-51a2ac9aa47f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:08.149426 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6a8af-bad1-445c-8ca3-56f545a5f80d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:08.173507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:08.187186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b517440-2d6f-46fe-b2fe-ee6c0aa4295b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:08.187845 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b517440-2d6f-46fe-b2fe-ee6c0aa4295b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:08.188136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b517440-2d6f-46fe-b2fe-ee6c0aa4295b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:08.188415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b517440-2d6f-46fe-b2fe-ee6c0aa4295b None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:08.189168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b517440-2d6f-46fe-b2fe-ee6c0aa4295b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:08.189475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b517440-2d6f-46fe-b2fe-ee6c0aa4295b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:08.190885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b517440-2d6f-46fe-b2fe-ee6c0aa4295b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:08.207311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 561/1078] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:08.208200 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b517440-2d6f-46fe-b2fe-ee6c0aa4295b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:08.264305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 518/1079] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:43:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:43:08.269796 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4f105-c99d-4df7-953f-51a2ac9aa47f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:08.282546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:08.317899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b517440-2d6f-46fe-b2fe-ee6c0aa4295b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:08.354333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445fb35-bb67-4f6b-8864-55b753b73693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:08.355192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445fb35-bb67-4f6b-8864-55b753b73693 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:08.355485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445fb35-bb67-4f6b-8864-55b753b73693 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:08.355768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c445fb35-bb67-4f6b-8864-55b753b73693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:08.363421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445fb35-bb67-4f6b-8864-55b753b73693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:08.365007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c445fb35-bb67-4f6b-8864-55b753b73693 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:08.380342 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4f105-c99d-4df7-953f-51a2ac9aa47f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c445fb35-bb67-4f6b-8864-55b753b73693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:43:08.380932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:08.381839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:08.382759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:43:08.382759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:43:08.382759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:43:08.382759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:43:08.382759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:43:08.382759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:43:08.382759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:43:08.382759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:43:08.382759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 562/1080] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:43:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:08.387885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445fb35-bb67-4f6b-8864-55b753b73693 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:08.407597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:08.423809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c61ad-dfb5-4cd0-b721-8603673de177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:08.424602 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c61ad-dfb5-4cd0-b721-8603673de177 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:08.425013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c61ad-dfb5-4cd0-b721-8603673de177 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:08.425305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c61ad-dfb5-4cd0-b721-8603673de177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:08.436462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c61ad-dfb5-4cd0-b721-8603673de177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:08.437945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c61ad-dfb5-4cd0-b721-8603673de177 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f63c61ad-dfb5-4cd0-b721-8603673de177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:43:08.456537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:08.457771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:08.458670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:43:08.458670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:43:08.458670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:08.458670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:08.458670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 519/1081] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:08.463699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c61ad-dfb5-4cd0-b721-8603673de177 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:08.505478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c445fb35-bb67-4f6b-8864-55b753b73693 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:08.539280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:08.551059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bc5cb-4d4f-46ea-a16a-d79397ba16b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:08.552018 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bc5cb-4d4f-46ea-a16a-d79397ba16b8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:08.552640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bc5cb-4d4f-46ea-a16a-d79397ba16b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:08.553520 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bc5cb-4d4f-46ea-a16a-d79397ba16b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:08.554817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bc5cb-4d4f-46ea-a16a-d79397ba16b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:08.555492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bc5cb-4d4f-46ea-a16a-d79397ba16b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:08.557469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887bc5cb-4d4f-46ea-a16a-d79397ba16b8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:08.570566 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c61ad-dfb5-4cd0-b721-8603673de177 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:08.573911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 563/1082] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:43:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:08.580046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bc5cb-4d4f-46ea-a16a-d79397ba16b8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:08.608686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e590-34d6-473b-8a2e-0cff68d0c665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:08.609371 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e590-34d6-473b-8a2e-0cff68d0c665 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:08.609660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e590-34d6-473b-8a2e-0cff68d0c665 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:08.609944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e590-34d6-473b-8a2e-0cff68d0c665 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:08.610674 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2e590-34d6-473b-8a2e-0cff68d0c665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:08.610984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2e590-34d6-473b-8a2e-0cff68d0c665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:08.612425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2e590-34d6-473b-8a2e-0cff68d0c665 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:08.625437 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 520/1083] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:08.705845 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bc5cb-4d4f-46ea-a16a-d79397ba16b8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:08.743590 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff7efd-aeb1-4956-99e2-1223d09b325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:08.744498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7efd-aeb1-4956-99e2-1223d09b325a None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:08.744816 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7efd-aeb1-4956-99e2-1223d09b325a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:08.745098 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff7efd-aeb1-4956-99e2-1223d09b325a None admin] PATH_INFO: `/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/36c18bbd22f74efbae7aab1cdd3f6403/roles/370db8aebcaf44e995d905fdd73e0bbd` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:08.747470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff7efd-aeb1-4956-99e2-1223d09b325a None admin] RBAC: Authorizing `identity:create_grant(project_id=19c2f661cc934aa7869e9b3d4efa875e, user_id=36c18bbd22f74efbae7aab1cdd3f6403, role_id=370db8aebcaf44e995d905fdd73e0bbd)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:08.749007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff7efd-aeb1-4956-99e2-1223d09b325a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:08.765908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 564/1084] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:43:08 2022] PUT /identity/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/36c18bbd22f74efbae7aab1cdd3f6403/roles/370db8aebcaf44e995d905fdd73e0bbd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:09.316903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb77e4-0c1c-44bb-b1f4-f2b2b72c12d0 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:09.317465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb77e4-0c1c-44bb-b1f4-f2b2b72c12d0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:09.317465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbb77e4-0c1c-44bb-b1f4-f2b2b72c12d0 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:09.318813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 521/1085] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:09.323952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6c9e4-bb32-412b-8d1f-78bcd0304097 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:09.324335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6c9e4-bb32-412b-8d1f-78bcd0304097 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:09.324512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6c9e4-bb32-412b-8d1f-78bcd0304097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:09.340762 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2cf6c9e4-bb32-412b-8d1f-78bcd0304097 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:09.341968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2cf6c9e4-bb32-412b-8d1f-78bcd0304097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:09.386242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:09.412210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 565/1086] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:09.437513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb77e4-0c1c-44bb-b1f4-f2b2b72c12d0 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:09.470127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:09.544141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbb77e4-0c1c-44bb-b1f4-f2b2b72c12d0 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:09.577377 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df907941-af1f-496b-8acb-7bfb4903fd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:09.578105 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df907941-af1f-496b-8acb-7bfb4903fd95 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:09.578389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df907941-af1f-496b-8acb-7bfb4903fd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:09.578671 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df907941-af1f-496b-8acb-7bfb4903fd95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:09.589796 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df907941-af1f-496b-8acb-7bfb4903fd95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-df907941-af1f-496b-8acb-7bfb4903fd95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:43:09.597915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:09.599061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:09.600375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:09.600375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:09.600375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:09.600375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:09.600375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:43:09.600375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:43:09.600375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:43:09.600375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:43:09.600375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:43:09.600375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:43:09.600375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:09.600375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:09.604007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 522/1087] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:43:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:09.605341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6c9e4-bb32-412b-8d1f-78bcd0304097 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:09.702605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6c9e4-bb32-412b-8d1f-78bcd0304097 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:09.736311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8227b26-38cd-439f-be87-ca70ab6902e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:09.737012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8227b26-38cd-439f-be87-ca70ab6902e2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:09.737301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8227b26-38cd-439f-be87-ca70ab6902e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:09.737578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8227b26-38cd-439f-be87-ca70ab6902e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:09.738319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8227b26-38cd-439f-be87-ca70ab6902e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:09.738631 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8227b26-38cd-439f-be87-ca70ab6902e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:09.739994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8227b26-38cd-439f-be87-ca70ab6902e2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:09.748090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 566/1088] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:43:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:09.753798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df907941-af1f-496b-8acb-7bfb4903fd95 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:09.843927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:09.852848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df907941-af1f-496b-8acb-7bfb4903fd95 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:09.887669 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe41e8b-2bf6-482e-9f8a-113660d8d6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:09.888542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe41e8b-2bf6-482e-9f8a-113660d8d6cb None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:09.888854 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe41e8b-2bf6-482e-9f8a-113660d8d6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:09.889132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe41e8b-2bf6-482e-9f8a-113660d8d6cb None admin] PATH_INFO: `/v3/users/68da1e7aa7714d0488d1a8bf3245944b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:09.890320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe41e8b-2bf6-482e-9f8a-113660d8d6cb None admin] RBAC: Authorizing `identity:get_user(user_id=68da1e7aa7714d0488d1a8bf3245944b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:09.891739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe41e8b-2bf6-482e-9f8a-113660d8d6cb None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:09.893407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 523/1089] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:09 2022] GET /identity/v3/users/68da1e7aa7714d0488d1a8bf3245944b => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:09.899079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8227b26-38cd-439f-be87-ca70ab6902e2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:09.941334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:09.997519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8227b26-38cd-439f-be87-ca70ab6902e2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:10.030515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e030f-a782-44f5-a802-a75f4797e046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:10.031154 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e030f-a782-44f5-a802-a75f4797e046 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:10.031439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e030f-a782-44f5-a802-a75f4797e046 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:10.031715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e030f-a782-44f5-a802-a75f4797e046 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:10.097516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e030f-a782-44f5-a802-a75f4797e046 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:10.098884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e030f-a782-44f5-a802-a75f4797e046 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:10.136734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:10.202400 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 567/1090] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:43:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 11:43:10.208616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe41e8b-2bf6-482e-9f8a-113660d8d6cb None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:10.309057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe41e8b-2bf6-482e-9f8a-113660d8d6cb None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:10.343558 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c6f19-2615-4fb6-ba10-e235369eed1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:10.344444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c6f19-2615-4fb6-ba10-e235369eed1a None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:10.344731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c6f19-2615-4fb6-ba10-e235369eed1a None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:10.345017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c6f19-2615-4fb6-ba10-e235369eed1a None admin] PATH_INFO: `/v3/projects/9f97e0b2639b489098f7163c943324e8` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:10.346316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c6f19-2615-4fb6-ba10-e235369eed1a None admin] RBAC: Authorizing `identity:get_project(project_id=9f97e0b2639b489098f7163c943324e8)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:10.347818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c6f19-2615-4fb6-ba10-e235369eed1a None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:10.349719 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 524/1091] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:43:10 2022] GET /identity/v3/projects/9f97e0b2639b489098f7163c943324e8 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:10.355815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e030f-a782-44f5-a802-a75f4797e046 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:10.455160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e030f-a782-44f5-a802-a75f4797e046 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:10.489133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36dc99-758d-46ac-b958-142eaa5d919c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:10.490233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36dc99-758d-46ac-b958-142eaa5d919c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:10.490726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36dc99-758d-46ac-b958-142eaa5d919c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:10.491228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36dc99-758d-46ac-b958-142eaa5d919c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:10.492541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36dc99-758d-46ac-b958-142eaa5d919c None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=9f97e0b2639b489098f7163c943324e8, user.id=68da1e7aa7714d0488d1a8bf3245944b) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:10.492878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36dc99-758d-46ac-b958-142eaa5d919c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:10.494526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36dc99-758d-46ac-b958-142eaa5d919c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:10.501877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 568/1092] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:43:10 2022] GET /identity/v3/role_assignments?user.id=68da1e7aa7714d0488d1a8bf3245944b&scope.project.id=9f97e0b2639b489098f7163c943324e8&role.id=569707378794437b93d168fc8bb9cbc3 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:10.822473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09c72e-f42c-4f09-9af4-97e8ee171c9c None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:10.822914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09c72e-f42c-4f09-9af4-97e8ee171c9c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:10.823223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09c72e-f42c-4f09-9af4-97e8ee171c9c None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:10.824904 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 525/1093] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:10.829992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7e463-2aeb-4092-8ac3-07d9b667eb55 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:10.830282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7e463-2aeb-4092-8ac3-07d9b667eb55 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:10.830563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7e463-2aeb-4092-8ac3-07d9b667eb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:10.837171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:10.848357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-28b7e463-2aeb-4092-8ac3-07d9b667eb55 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:10.849579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-28b7e463-2aeb-4092-8ac3-07d9b667eb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:10.917158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 569/1094] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:10.948206 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09c72e-f42c-4f09-9af4-97e8ee171c9c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:10.976251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:11.049576 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09c72e-f42c-4f09-9af4-97e8ee171c9c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:11.089029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b8395-7bf2-4552-85b0-2b26dd2b59e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:11.089841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b8395-7bf2-4552-85b0-2b26dd2b59e0 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:11.090122 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b8395-7bf2-4552-85b0-2b26dd2b59e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:11.090396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b8395-7bf2-4552-85b0-2b26dd2b59e0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:11.101494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b8395-7bf2-4552-85b0-2b26dd2b59e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-035b8395-7bf2-4552-85b0-2b26dd2b59e0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:43:11.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:11.111307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:11.112616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:11.112616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:11.112616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:11.112616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:11.112616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:43:11.112616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:43:11.112616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:43:11.112616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:43:11.112616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:43:11.112616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:43:11.112616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 11:43:11.112616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:11.116266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 526/1095] 10.222.0.67 () {56 vars in 1120 bytes} [Thu Jun 23 11:43:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:11.117210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7e463-2aeb-4092-8ac3-07d9b667eb55 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:11.212497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7e463-2aeb-4092-8ac3-07d9b667eb55 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:11.248149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9209f-cfdf-4f47-a2b6-d56bd345e1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:11.248836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9209f-cfdf-4f47-a2b6-d56bd345e1aa None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:11.249121 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9209f-cfdf-4f47-a2b6-d56bd345e1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:11.249399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9209f-cfdf-4f47-a2b6-d56bd345e1aa None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:11.250137 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9209f-cfdf-4f47-a2b6-d56bd345e1aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:11.250446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9209f-cfdf-4f47-a2b6-d56bd345e1aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:11.251807 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9209f-cfdf-4f47-a2b6-d56bd345e1aa None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:11.253207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:11.261497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 570/1096] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 23 11:43:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:11.267043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b8395-7bf2-4552-85b0-2b26dd2b59e0 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:11.354530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:11.368140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b8395-7bf2-4552-85b0-2b26dd2b59e0 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:11.400886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60439d4-af72-469f-9ff0-0caca5fc4b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:11.401662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60439d4-af72-469f-9ff0-0caca5fc4b38 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:11.401944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60439d4-af72-469f-9ff0-0caca5fc4b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:11.402218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60439d4-af72-469f-9ff0-0caca5fc4b38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:11.413270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60439d4-af72-469f-9ff0-0caca5fc4b38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:11.414695 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60439d4-af72-469f-9ff0-0caca5fc4b38 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-d60439d4-af72-469f-9ff0-0caca5fc4b38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:43:11.431881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:11.432852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:11.433782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:43:11.433782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:43:11.433782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:11.433782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:11.433782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 527/1097] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:11.438739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9209f-cfdf-4f47-a2b6-d56bd345e1aa None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:11.538677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9209f-cfdf-4f47-a2b6-d56bd345e1aa None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:11.571364 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5277470-1772-4ce4-8a72-10346d558df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:11.572022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5277470-1772-4ce4-8a72-10346d558df3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:11.572309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5277470-1772-4ce4-8a72-10346d558df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:11.572588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5277470-1772-4ce4-8a72-10346d558df3 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:11.573362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5277470-1772-4ce4-8a72-10346d558df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:11.573678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5277470-1772-4ce4-8a72-10346d558df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:11.575039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5277470-1772-4ce4-8a72-10346d558df3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:11.587056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 571/1098] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:11.593164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60439d4-af72-469f-9ff0-0caca5fc4b38 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:11.696295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60439d4-af72-469f-9ff0-0caca5fc4b38 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:11.729975 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f424c0-59ca-441f-b377-8f568dcf961b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:11.730760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f424c0-59ca-441f-b377-8f568dcf961b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:11.731046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f424c0-59ca-441f-b377-8f568dcf961b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:11.731325 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f424c0-59ca-441f-b377-8f568dcf961b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:11.739605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f424c0-59ca-441f-b377-8f568dcf961b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:11.741075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f424c0-59ca-441f-b377-8f568dcf961b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:11.743549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-17f424c0-59ca-441f-b377-8f568dcf961b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:43:11.755728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:11.756773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:11.757762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:11.757762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:11.757762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:43:11.757762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:43:11.757762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:43:11.757762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:43:11.757762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:43:11.757762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:43:11.757762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 11:43:11.757762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:11.757762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 528/1099] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 23 11:43:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:43:11.762699 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5277470-1772-4ce4-8a72-10346d558df3 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:11.819081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:11.861007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5277470-1772-4ce4-8a72-10346d558df3 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:11.894246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8d18a-d850-4f39-b661-1cbb2f318cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:11.894900 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8d18a-d850-4f39-b661-1cbb2f318cca None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:11.895185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8d18a-d850-4f39-b661-1cbb2f318cca None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:11.895463 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8d18a-d850-4f39-b661-1cbb2f318cca None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:11.896278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8d18a-d850-4f39-b661-1cbb2f318cca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:11.896587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8d18a-d850-4f39-b661-1cbb2f318cca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:11.898019 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8d18a-d850-4f39-b661-1cbb2f318cca None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:11.914630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 572/1100] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 23 11:43:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:11.920382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f424c0-59ca-441f-b377-8f568dcf961b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:12.017689 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f424c0-59ca-441f-b377-8f568dcf961b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:12.049721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0006d44-20c2-4b00-902d-aeba36d4a93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:12.050349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0006d44-20c2-4b00-902d-aeba36d4a93b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:12.050630 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0006d44-20c2-4b00-902d-aeba36d4a93b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:12.050905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0006d44-20c2-4b00-902d-aeba36d4a93b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:12.115209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:12.117674 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0006d44-20c2-4b00-902d-aeba36d4a93b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:12.119451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0006d44-20c2-4b00-902d-aeba36d4a93b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:12.217166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 529/1101] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:43:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:43:12.222808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8d18a-d850-4f39-b661-1cbb2f318cca None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:12.322745 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8d18a-d850-4f39-b661-1cbb2f318cca None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:12.336534 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:12.358050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc274d0d-e18b-417e-9aaa-57184188eaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:12.358837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc274d0d-e18b-417e-9aaa-57184188eaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:12.359119 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc274d0d-e18b-417e-9aaa-57184188eaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:12.359392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc274d0d-e18b-417e-9aaa-57184188eaa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:12.370002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc274d0d-e18b-417e-9aaa-57184188eaa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:12.371398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc274d0d-e18b-417e-9aaa-57184188eaa0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-bc274d0d-e18b-417e-9aaa-57184188eaa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:43:12.389732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:12.390812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:12.391743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:43:12.391743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:43:12.391743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:43:12.391743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:43:12.391743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 573/1102] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:12.396576 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0006d44-20c2-4b00-902d-aeba36d4a93b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:12.490405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:12.493484 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0006d44-20c2-4b00-902d-aeba36d4a93b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:12.527290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c9021-930c-4ff7-a6a9-cf0b4a70af8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:12.527955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c9021-930c-4ff7-a6a9-cf0b4a70af8e None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:12.528380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c9021-930c-4ff7-a6a9-cf0b4a70af8e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:12.528519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c9021-930c-4ff7-a6a9-cf0b4a70af8e None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:12.529297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c9021-930c-4ff7-a6a9-cf0b4a70af8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:12.529620 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c9021-930c-4ff7-a6a9-cf0b4a70af8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:12.531141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c9021-930c-4ff7-a6a9-cf0b4a70af8e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:12.544086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 530/1103] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:12.550165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc274d0d-e18b-417e-9aaa-57184188eaa0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:12.581655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d7717-6d46-42db-a608-b93fb32bff33 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:12.581950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d7717-6d46-42db-a608-b93fb32bff33 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:12.582237 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d7717-6d46-42db-a608-b93fb32bff33 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:12.583616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 531/1104] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:43:12.588483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950df6e-5167-4b37-be2a-45cb96f4af36 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:12.588889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950df6e-5167-4b37-be2a-45cb96f4af36 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:12.589178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950df6e-5167-4b37-be2a-45cb96f4af36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:12.605927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c950df6e-5167-4b37-be2a-45cb96f4af36 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:12.607742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c950df6e-5167-4b37-be2a-45cb96f4af36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:12.650762 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc274d0d-e18b-417e-9aaa-57184188eaa0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:12.677309 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 532/1105] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:43:12.695676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb64ae4-1655-4b65-8ef9-b330f3ea0909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:12.696468 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb64ae4-1655-4b65-8ef9-b330f3ea0909 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:12.696782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb64ae4-1655-4b65-8ef9-b330f3ea0909 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:12.697069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb64ae4-1655-4b65-8ef9-b330f3ea0909 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:12.701595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950df6e-5167-4b37-be2a-45cb96f4af36 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:12.704042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb64ae4-1655-4b65-8ef9-b330f3ea0909 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:12.705540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb64ae4-1655-4b65-8ef9-b330f3ea0909 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-feb64ae4-1655-4b65-8ef9-b330f3ea0909 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:43:12.720275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:12.721322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:12.722318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:43:12.722318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:43:12.722318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:43:12.722318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:43:12.722318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:43:12.722318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:43:12.722318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:43:12.722318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:43:12.722318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 574/1106] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:43:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:12.727404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb64ae4-1655-4b65-8ef9-b330f3ea0909 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:12.755331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:12.790578 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:12.801009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950df6e-5167-4b37-be2a-45cb96f4af36 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:12.828059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb64ae4-1655-4b65-8ef9-b330f3ea0909 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:12.834209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663c1e3-2a81-47ba-bfb2-d7bb55009430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:12.834999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663c1e3-2a81-47ba-bfb2-d7bb55009430 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:12.835278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663c1e3-2a81-47ba-bfb2-d7bb55009430 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:12.835548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663c1e3-2a81-47ba-bfb2-d7bb55009430 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:12.836740 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663c1e3-2a81-47ba-bfb2-d7bb55009430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:12.838159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663c1e3-2a81-47ba-bfb2-d7bb55009430 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:12.839765 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 533/1107] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:12.846177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663c1e3-2a81-47ba-bfb2-d7bb55009430 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:12.867960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c18b1-c996-4194-b7ae-82f85175efa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:12.868665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c18b1-c996-4194-b7ae-82f85175efa7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:12.868975 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c18b1-c996-4194-b7ae-82f85175efa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:12.869253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c18b1-c996-4194-b7ae-82f85175efa7 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:12.870033 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c18b1-c996-4194-b7ae-82f85175efa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:12.870346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c18b1-c996-4194-b7ae-82f85175efa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:12.871775 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c18b1-c996-4194-b7ae-82f85175efa7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:12.883879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 575/1108] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:43:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:12.889556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c18b1-c996-4194-b7ae-82f85175efa7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:12.949124 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663c1e3-2a81-47ba-bfb2-d7bb55009430 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:12.978508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:12.985074 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c007a9-28c1-4ceb-9645-ce2557050e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:12.985744 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c007a9-28c1-4ceb-9645-ce2557050e56 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:12.986024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c007a9-28c1-4ceb-9645-ce2557050e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:12.986309 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c007a9-28c1-4ceb-9645-ce2557050e56 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:12.987028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c007a9-28c1-4ceb-9645-ce2557050e56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:12.988433 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c007a9-28c1-4ceb-9645-ce2557050e56 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:12.994939 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c18b1-c996-4194-b7ae-82f85175efa7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:13.009100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 534/1109] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:43:12 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:13.034003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d991f77-eea4-49b1-bd01-14bf9fb1b21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:13.034682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d991f77-eea4-49b1-bd01-14bf9fb1b21c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:13.034977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d991f77-eea4-49b1-bd01-14bf9fb1b21c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:13.035258 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d991f77-eea4-49b1-bd01-14bf9fb1b21c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:13.036120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d991f77-eea4-49b1-bd01-14bf9fb1b21c None admin] RBAC: Adding query filter params (role.id=370db8aebcaf44e995d905fdd73e0bbd, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=36c18bbd22f74efbae7aab1cdd3f6403) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:13.036436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d991f77-eea4-49b1-bd01-14bf9fb1b21c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:13.037987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d991f77-eea4-49b1-bd01-14bf9fb1b21c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:13.045282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 576/1110] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:43:12 2022] GET /identity/v3/role_assignments?user.id=36c18bbd22f74efbae7aab1cdd3f6403&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=370db8aebcaf44e995d905fdd73e0bbd => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:15.109375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1def7be-77d3-4f65-a0cf-6e57705b91e8 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:15.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1def7be-77d3-4f65-a0cf-6e57705b91e8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:15.110168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1def7be-77d3-4f65-a0cf-6e57705b91e8 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:15.112093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 535/1111] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:15.118672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889cfc8-3032-4603-b40d-92d976efaa5f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:15.119199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889cfc8-3032-4603-b40d-92d976efaa5f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:15.119693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889cfc8-3032-4603-b40d-92d976efaa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:15.140768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a889cfc8-3032-4603-b40d-92d976efaa5f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:15.141956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a889cfc8-3032-4603-b40d-92d976efaa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:15.205908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 577/1112] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:15.229051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1def7be-77d3-4f65-a0cf-6e57705b91e8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:15.245616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:15.329691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1def7be-77d3-4f65-a0cf-6e57705b91e8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:15.362829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd4245-25de-497a-84d5-4a986cef0df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:15.363615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd4245-25de-497a-84d5-4a986cef0df7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:15.363903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd4245-25de-497a-84d5-4a986cef0df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:15.364183 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd4245-25de-497a-84d5-4a986cef0df7 None admin] PATH_INFO: `/v3/roles/2d0e0231af5d4e51b80de6bb65e8b02b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:15.365783 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd4245-25de-497a-84d5-4a986cef0df7 None admin] RBAC: Authorizing `identity:get_role(role_id=2d0e0231af5d4e51b80de6bb65e8b02b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:15.367076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd4245-25de-497a-84d5-4a986cef0df7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:15.368566 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 536/1113] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:15 2022] GET /identity/v3/roles/2d0e0231af5d4e51b80de6bb65e8b02b => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:15.375508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889cfc8-3032-4603-b40d-92d976efaa5f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:15.454192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:15.473123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889cfc8-3032-4603-b40d-92d976efaa5f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:15.506125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e39ba-e4ea-419b-9d14-3ff5431c706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:15.506983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e39ba-e4ea-419b-9d14-3ff5431c706d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:15.507271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e39ba-e4ea-419b-9d14-3ff5431c706d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:15.507551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e39ba-e4ea-419b-9d14-3ff5431c706d None admin] PATH_INFO: `/v3/users/b7d8686ac0104dbd964191ccd5eab54a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:15.520992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e39ba-e4ea-419b-9d14-3ff5431c706d None admin] RBAC: Authorizing `identity:get_user(user_id=b7d8686ac0104dbd964191ccd5eab54a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:15.522440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133e39ba-e4ea-419b-9d14-3ff5431c706d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:15.524087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 578/1114] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:15 2022] GET /identity/v3/users/b7d8686ac0104dbd964191ccd5eab54a => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:15.529841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd4245-25de-497a-84d5-4a986cef0df7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:15.620041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:15.637877 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd4245-25de-497a-84d5-4a986cef0df7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:15.671087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd1734-9a4b-4813-9699-78c3a32ba721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:15.671737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd1734-9a4b-4813-9699-78c3a32ba721 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:15.672025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd1734-9a4b-4813-9699-78c3a32ba721 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:15.672304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd1734-9a4b-4813-9699-78c3a32ba721 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:15.737301 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd1734-9a4b-4813-9699-78c3a32ba721 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:15.738745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd1734-9a4b-4813-9699-78c3a32ba721 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:15.816291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:15.835986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 537/1115] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:43:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:43:15.841919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e39ba-e4ea-419b-9d14-3ff5431c706d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:15.941717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e39ba-e4ea-419b-9d14-3ff5431c706d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:15.975476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e13b5-b229-40ff-95c8-b24304a86140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:15.976324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e13b5-b229-40ff-95c8-b24304a86140 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:15.976611 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e13b5-b229-40ff-95c8-b24304a86140 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:15.976919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e13b5-b229-40ff-95c8-b24304a86140 None admin] PATH_INFO: `/v3/projects/9f97e0b2639b489098f7163c943324e8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:15.978097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e13b5-b229-40ff-95c8-b24304a86140 None admin] RBAC: Authorizing `identity:get_project(project_id=9f97e0b2639b489098f7163c943324e8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:15.979529 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e13b5-b229-40ff-95c8-b24304a86140 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:15.981309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 579/1116] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:43:15 2022] GET /identity/v3/projects/9f97e0b2639b489098f7163c943324e8 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:15.987782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd1734-9a4b-4813-9699-78c3a32ba721 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:16.083677 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd1734-9a4b-4813-9699-78c3a32ba721 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:16.116745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da1c63-c2a1-4b9d-8b08-8a34b8feaf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:16.117388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da1c63-c2a1-4b9d-8b08-8a34b8feaf77 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:16.117676 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da1c63-c2a1-4b9d-8b08-8a34b8feaf77 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:16.117956 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da1c63-c2a1-4b9d-8b08-8a34b8feaf77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:16.118785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da1c63-c2a1-4b9d-8b08-8a34b8feaf77 None admin] RBAC: Adding query filter params (role.id=2d0e0231af5d4e51b80de6bb65e8b02b, scope.project.id=9f97e0b2639b489098f7163c943324e8, user.id=b7d8686ac0104dbd964191ccd5eab54a) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:16.119097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da1c63-c2a1-4b9d-8b08-8a34b8feaf77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:16.120517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da1c63-c2a1-4b9d-8b08-8a34b8feaf77 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:16.126488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 538/1117] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:43:15 2022] GET /identity/v3/role_assignments?user.id=b7d8686ac0104dbd964191ccd5eab54a&scope.project.id=9f97e0b2639b489098f7163c943324e8&role.id=2d0e0231af5d4e51b80de6bb65e8b02b => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:18.121674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38e60b-d1d8-4cb0-85c0-7393034e09c3 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:18.122228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38e60b-d1d8-4cb0-85c0-7393034e09c3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:18.122228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38e60b-d1d8-4cb0-85c0-7393034e09c3 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:18.123576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 580/1118] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:18.128832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b628dd-c59c-4b19-9976-983bfeb34953 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:18.129262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b628dd-c59c-4b19-9976-983bfeb34953 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:18.129399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b628dd-c59c-4b19-9976-983bfeb34953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:18.145663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-86b628dd-c59c-4b19-9976-983bfeb34953 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:18.146879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-86b628dd-c59c-4b19-9976-983bfeb34953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:18.188878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:18.213586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 539/1119] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:18.237431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38e60b-d1d8-4cb0-85c0-7393034e09c3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:18.246686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:18.336715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38e60b-d1d8-4cb0-85c0-7393034e09c3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:18.369686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bc63f-c206-4179-b9d0-058e54f89a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:18.370539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bc63f-c206-4179-b9d0-058e54f89a2c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:18.370828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bc63f-c206-4179-b9d0-058e54f89a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:18.371108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bc63f-c206-4179-b9d0-058e54f89a2c None admin] PATH_INFO: `/v3/roles/2d0e0231af5d4e51b80de6bb65e8b02b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:18.373272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bc63f-c206-4179-b9d0-058e54f89a2c None admin] RBAC: Authorizing `identity:get_role(role_id=2d0e0231af5d4e51b80de6bb65e8b02b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:18.374597 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bc63f-c206-4179-b9d0-058e54f89a2c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:18.375980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 581/1120] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:18 2022] GET /identity/v3/roles/2d0e0231af5d4e51b80de6bb65e8b02b => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:18.382077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b628dd-c59c-4b19-9976-983bfeb34953 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:18.476888 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b628dd-c59c-4b19-9976-983bfeb34953 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:18.509624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0ee0f-8495-4b8a-8b59-97e22f21517c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:18.510478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0ee0f-8495-4b8a-8b59-97e22f21517c None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:18.510757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0ee0f-8495-4b8a-8b59-97e22f21517c None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:18.511028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0ee0f-8495-4b8a-8b59-97e22f21517c None admin] PATH_INFO: `/v3/users/b7d8686ac0104dbd964191ccd5eab54a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:18.512211 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0ee0f-8495-4b8a-8b59-97e22f21517c None admin] RBAC: Authorizing `identity:get_user(user_id=b7d8686ac0104dbd964191ccd5eab54a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:18.513693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0ee0f-8495-4b8a-8b59-97e22f21517c None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:18.515331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 540/1121] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:18 2022] GET /identity/v3/users/b7d8686ac0104dbd964191ccd5eab54a => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:18.521300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bc63f-c206-4179-b9d0-058e54f89a2c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:18.595039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:18.620103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bc63f-c206-4179-b9d0-058e54f89a2c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:18.652023 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ae0cf-936e-416e-ba7a-516e082ff8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:18.652850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ae0cf-936e-416e-ba7a-516e082ff8d6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:18.653143 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ae0cf-936e-416e-ba7a-516e082ff8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:18.653425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ae0cf-936e-416e-ba7a-516e082ff8d6 None admin] PATH_INFO: `/v3/projects/9f97e0b2639b489098f7163c943324e8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:18.655127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ae0cf-936e-416e-ba7a-516e082ff8d6 None admin] RBAC: Authorizing `identity:get_project(project_id=9f97e0b2639b489098f7163c943324e8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:18.656571 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ae0cf-936e-416e-ba7a-516e082ff8d6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:18.658583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 582/1122] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:43:18 2022] GET /identity/v3/projects/9f97e0b2639b489098f7163c943324e8 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:18.664530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0ee0f-8495-4b8a-8b59-97e22f21517c None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:18.710649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:18.762985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0ee0f-8495-4b8a-8b59-97e22f21517c None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:18.796075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426708a-3f69-4895-9b36-9a0e0048a93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:18.796988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426708a-3f69-4895-9b36-9a0e0048a93e None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:18.797277 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426708a-3f69-4895-9b36-9a0e0048a93e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:18.797556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426708a-3f69-4895-9b36-9a0e0048a93e None admin] PATH_INFO: `/v3/projects/9f97e0b2639b489098f7163c943324e8/users/b7d8686ac0104dbd964191ccd5eab54a/roles/2d0e0231af5d4e51b80de6bb65e8b02b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:18.799784 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426708a-3f69-4895-9b36-9a0e0048a93e None admin] RBAC: Authorizing `identity:create_grant(project_id=9f97e0b2639b489098f7163c943324e8, user_id=b7d8686ac0104dbd964191ccd5eab54a, role_id=2d0e0231af5d4e51b80de6bb65e8b02b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:18.801248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426708a-3f69-4895-9b36-9a0e0048a93e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:18.813949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 541/1123] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:43:18 2022] PUT /identity/v3/projects/9f97e0b2639b489098f7163c943324e8/users/b7d8686ac0104dbd964191ccd5eab54a/roles/2d0e0231af5d4e51b80de6bb65e8b02b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:43:20.811720 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbd2ec-4c53-4db1-ae82-8e26f54d7ba2 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:20.812270 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbd2ec-4c53-4db1-ae82-8e26f54d7ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:20.812270 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbd2ec-4c53-4db1-ae82-8e26f54d7ba2 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:20.813703 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 583/1124] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:20.818848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86dc97-0004-45f9-b59c-5108ec0468eb None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:20.819262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86dc97-0004-45f9-b59c-5108ec0468eb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:20.819405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86dc97-0004-45f9-b59c-5108ec0468eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:20.836561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6e86dc97-0004-45f9-b59c-5108ec0468eb None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:20.837787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6e86dc97-0004-45f9-b59c-5108ec0468eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:20.901452 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 542/1125] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:20.926345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbd2ec-4c53-4db1-ae82-8e26f54d7ba2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:21.026585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbd2ec-4c53-4db1-ae82-8e26f54d7ba2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:21.060509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd708ed-37b9-4cd3-b288-d8b6b1b4293a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:21.061376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd708ed-37b9-4cd3-b288-d8b6b1b4293a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:21.061659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd708ed-37b9-4cd3-b288-d8b6b1b4293a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:21.061935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd708ed-37b9-4cd3-b288-d8b6b1b4293a None admin] PATH_INFO: `/v3/roles/2d0e0231af5d4e51b80de6bb65e8b02b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:21.063391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd708ed-37b9-4cd3-b288-d8b6b1b4293a None admin] RBAC: Authorizing `identity:get_role(role_id=2d0e0231af5d4e51b80de6bb65e8b02b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:21.064770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd708ed-37b9-4cd3-b288-d8b6b1b4293a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:21.066139 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 584/1126] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:20 2022] GET /identity/v3/roles/2d0e0231af5d4e51b80de6bb65e8b02b => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:21.071896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86dc97-0004-45f9-b59c-5108ec0468eb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:21.104298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:21.172037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86dc97-0004-45f9-b59c-5108ec0468eb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:21.205269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781df184-3dbc-4c60-8327-bd530d8f8b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:21.206072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781df184-3dbc-4c60-8327-bd530d8f8b71 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:21.206360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781df184-3dbc-4c60-8327-bd530d8f8b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:21.206641 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781df184-3dbc-4c60-8327-bd530d8f8b71 None admin] PATH_INFO: `/v3/users/b7d8686ac0104dbd964191ccd5eab54a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:21.207830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781df184-3dbc-4c60-8327-bd530d8f8b71 None admin] RBAC: Authorizing `identity:get_user(user_id=b7d8686ac0104dbd964191ccd5eab54a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:21.209299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781df184-3dbc-4c60-8327-bd530d8f8b71 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:21.210934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 543/1127] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:21 2022] GET /identity/v3/users/b7d8686ac0104dbd964191ccd5eab54a => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:21.217323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd708ed-37b9-4cd3-b288-d8b6b1b4293a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:21.229251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:21.313409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd708ed-37b9-4cd3-b288-d8b6b1b4293a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:21.347673 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b8830-0695-48ee-aa0b-56a2cba79c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:21.348330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b8830-0695-48ee-aa0b-56a2cba79c84 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:21.348620 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b8830-0695-48ee-aa0b-56a2cba79c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:21.348932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b8830-0695-48ee-aa0b-56a2cba79c84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:21.413411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b8830-0695-48ee-aa0b-56a2cba79c84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:21.414856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b8830-0695-48ee-aa0b-56a2cba79c84 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:21.425291 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:21.508488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 585/1128] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:43:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:43:21.514851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781df184-3dbc-4c60-8327-bd530d8f8b71 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:21.611153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781df184-3dbc-4c60-8327-bd530d8f8b71 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:21.612780 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:21.652853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b149e9-c273-4aaa-868c-71087e6f0ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:21.653706 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b149e9-c273-4aaa-868c-71087e6f0ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:21.654018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b149e9-c273-4aaa-868c-71087e6f0ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:21.654295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b149e9-c273-4aaa-868c-71087e6f0ba1 None admin] PATH_INFO: `/v3/projects/9f97e0b2639b489098f7163c943324e8` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:21.655484 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b149e9-c273-4aaa-868c-71087e6f0ba1 None admin] RBAC: Authorizing `identity:get_project(project_id=9f97e0b2639b489098f7163c943324e8)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:21.656962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b149e9-c273-4aaa-868c-71087e6f0ba1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:21.658947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 544/1129] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:43:21 2022] GET /identity/v3/projects/9f97e0b2639b489098f7163c943324e8 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:21.665019 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b8830-0695-48ee-aa0b-56a2cba79c84 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:21.760653 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b8830-0695-48ee-aa0b-56a2cba79c84 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:21.771552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:21.794570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f100c-79e8-4378-b0bb-46220934deba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:21.795225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f100c-79e8-4378-b0bb-46220934deba None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:21.795512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f100c-79e8-4378-b0bb-46220934deba None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:21.795819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f100c-79e8-4378-b0bb-46220934deba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:21.796640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f100c-79e8-4378-b0bb-46220934deba None admin] RBAC: Adding query filter params (role.id=2d0e0231af5d4e51b80de6bb65e8b02b, scope.project.id=9f97e0b2639b489098f7163c943324e8, user.id=b7d8686ac0104dbd964191ccd5eab54a) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:21.796984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f100c-79e8-4378-b0bb-46220934deba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:21.798407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f100c-79e8-4378-b0bb-46220934deba None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:21.804111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 586/1130] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:43:21 2022] GET /identity/v3/role_assignments?user.id=b7d8686ac0104dbd964191ccd5eab54a&scope.project.id=9f97e0b2639b489098f7163c943324e8&role.id=2d0e0231af5d4e51b80de6bb65e8b02b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:23.877497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a56c7-ef7d-446c-815e-b89907f9116c None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:23.878062 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a56c7-ef7d-446c-815e-b89907f9116c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:23.878062 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a56c7-ef7d-446c-815e-b89907f9116c None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:23.879436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 545/1131] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:23.884858 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663186f-21bb-42b9-8f52-4b388c7d5253 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:23.885295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663186f-21bb-42b9-8f52-4b388c7d5253 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:23.885416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663186f-21bb-42b9-8f52-4b388c7d5253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:23.902951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4663186f-21bb-42b9-8f52-4b388c7d5253 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:23.904656 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4663186f-21bb-42b9-8f52-4b388c7d5253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:23.968406 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 587/1132] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:23.992314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a56c7-ef7d-446c-815e-b89907f9116c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:24.089553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a56c7-ef7d-446c-815e-b89907f9116c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:24.124227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009baae8-650c-4567-a2f0-e71cb9d811f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:24.125124 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009baae8-650c-4567-a2f0-e71cb9d811f2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:24.125409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009baae8-650c-4567-a2f0-e71cb9d811f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:24.125686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009baae8-650c-4567-a2f0-e71cb9d811f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:24.126858 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009baae8-650c-4567-a2f0-e71cb9d811f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:24.128280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009baae8-650c-4567-a2f0-e71cb9d811f2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:24.129930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 546/1133] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:24.135759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663186f-21bb-42b9-8f52-4b388c7d5253 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:24.232799 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663186f-21bb-42b9-8f52-4b388c7d5253 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:24.235079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:24.267147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2525450f-cc73-46e1-9e51-1915fe431bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:24.267806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525450f-cc73-46e1-9e51-1915fe431bd4 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:24.268090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525450f-cc73-46e1-9e51-1915fe431bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:24.268373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2525450f-cc73-46e1-9e51-1915fe431bd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:24.269115 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525450f-cc73-46e1-9e51-1915fe431bd4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:24.270633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2525450f-cc73-46e1-9e51-1915fe431bd4 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:24.295932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 588/1134] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:43:24 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:26.383770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba18d95-8935-48e8-9d75-0ba8992ee6a2 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:26.384389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba18d95-8935-48e8-9d75-0ba8992ee6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:26.384389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba18d95-8935-48e8-9d75-0ba8992ee6a2 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:26.385790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 547/1135] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:26.392264 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8043e8-dfcb-4926-9da8-7d0ae8bc3e6b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:26.392777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8043e8-dfcb-4926-9da8-7d0ae8bc3e6b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:26.392883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8043e8-dfcb-4926-9da8-7d0ae8bc3e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:26.410067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2a8043e8-dfcb-4926-9da8-7d0ae8bc3e6b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:26.411257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2a8043e8-dfcb-4926-9da8-7d0ae8bc3e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:26.478083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 589/1136] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:43:26.502859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba18d95-8935-48e8-9d75-0ba8992ee6a2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:26.543692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:26.602795 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba18d95-8935-48e8-9d75-0ba8992ee6a2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:26.635262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ec268-1a63-436e-b6e3-44c3ac4411f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:26.635978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ec268-1a63-436e-b6e3-44c3ac4411f8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:26.636266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ec268-1a63-436e-b6e3-44c3ac4411f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:26.636547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ec268-1a63-436e-b6e3-44c3ac4411f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:26.637746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ec268-1a63-436e-b6e3-44c3ac4411f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:26.639716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ec268-1a63-436e-b6e3-44c3ac4411f8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:26.641361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 548/1137] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:43:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:26.647075 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8043e8-dfcb-4926-9da8-7d0ae8bc3e6b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:26.733380 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:26.748288 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8043e8-dfcb-4926-9da8-7d0ae8bc3e6b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:26.782694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276343d6-cad9-43a3-bbe6-ee65fe69fbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:26.783611 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276343d6-cad9-43a3-bbe6-ee65fe69fbf7 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:26.783902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276343d6-cad9-43a3-bbe6-ee65fe69fbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:26.784186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276343d6-cad9-43a3-bbe6-ee65fe69fbf7 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:26.784923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276343d6-cad9-43a3-bbe6-ee65fe69fbf7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:26.786363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276343d6-cad9-43a3-bbe6-ee65fe69fbf7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:26.806639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 590/1138] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:43:26 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:28.886950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad4209-9328-46d0-b805-2bad36620b63 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:28.887475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad4209-9328-46d0-b805-2bad36620b63 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:28.887585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad4209-9328-46d0-b805-2bad36620b63 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:28.888906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 549/1139] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:28.894004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc9d56-171f-4273-bbec-cb35a58bd8d2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:28.894455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc9d56-171f-4273-bbec-cb35a58bd8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:28.894558 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc9d56-171f-4273-bbec-cb35a58bd8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:28.911166 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-23bc9d56-171f-4273-bbec-cb35a58bd8d2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:28.912363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-23bc9d56-171f-4273-bbec-cb35a58bd8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:28.977548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 591/1140] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:29.001889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad4209-9328-46d0-b805-2bad36620b63 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:29.104902 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ad4209-9328-46d0-b805-2bad36620b63 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:29.113274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:29.139826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9dfb1-2b4e-42dc-82a5-cfdd777147d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:29.140606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9dfb1-2b4e-42dc-82a5-cfdd777147d2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:29.140921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9dfb1-2b4e-42dc-82a5-cfdd777147d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:29.141202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9dfb1-2b4e-42dc-82a5-cfdd777147d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:29.151080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9dfb1-2b4e-42dc-82a5-cfdd777147d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-e0d9dfb1-2b4e-42dc-82a5-cfdd777147d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:43:29.159479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:29.160661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:29.161884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 550/1141] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:43:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:29.166619 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc9d56-171f-4273-bbec-cb35a58bd8d2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:29.235208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:29.267727 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc9d56-171f-4273-bbec-cb35a58bd8d2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:29.302213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87edf3-b8d9-47b8-92af-0e7f5e648047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:29.302876 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87edf3-b8d9-47b8-92af-0e7f5e648047 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:29.303158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87edf3-b8d9-47b8-92af-0e7f5e648047 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:29.303433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87edf3-b8d9-47b8-92af-0e7f5e648047 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:29.304176 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87edf3-b8d9-47b8-92af-0e7f5e648047 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:29.304492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87edf3-b8d9-47b8-92af-0e7f5e648047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:29.305890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf87edf3-b8d9-47b8-92af-0e7f5e648047 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:29.313908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 592/1142] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:43:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:29.319626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9dfb1-2b4e-42dc-82a5-cfdd777147d2 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:29.418253 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9dfb1-2b4e-42dc-82a5-cfdd777147d2 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:29.451449 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b76a7-6d93-485f-a4b5-6b7a35fae649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:29.452313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b76a7-6d93-485f-a4b5-6b7a35fae649 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:29.452598 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b76a7-6d93-485f-a4b5-6b7a35fae649 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:29.452906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b76a7-6d93-485f-a4b5-6b7a35fae649 None admin] PATH_INFO: `/v3/users/e5cd3c8a10634d1e98401cfb47a922d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:29.465761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b76a7-6d93-485f-a4b5-6b7a35fae649 None admin] RBAC: Authorizing `identity:get_user(user_id=e5cd3c8a10634d1e98401cfb47a922d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:29.467846 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b76a7-6d93-485f-a4b5-6b7a35fae649 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:29.469582 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 551/1143] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:29 2022] GET /identity/v3/users/e5cd3c8a10634d1e98401cfb47a922d1 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:29.475424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87edf3-b8d9-47b8-92af-0e7f5e648047 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:29.572563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf87edf3-b8d9-47b8-92af-0e7f5e648047 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:29.605870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc9dd7-004f-4950-a311-9b7d02f88aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:29.606502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc9dd7-004f-4950-a311-9b7d02f88aae None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:29.606812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc9dd7-004f-4950-a311-9b7d02f88aae None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:29.607094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc9dd7-004f-4950-a311-9b7d02f88aae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:29.609737 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:29.673812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc9dd7-004f-4950-a311-9b7d02f88aae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:29.675303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc9dd7-004f-4950-a311-9b7d02f88aae None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:29.773054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 593/1144] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:43:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:43:29.779572 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b76a7-6d93-485f-a4b5-6b7a35fae649 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:29.804449 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:29.877053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b76a7-6d93-485f-a4b5-6b7a35fae649 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:29.916256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f89983-10d5-417c-a512-513f7fadd0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:29.917143 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f89983-10d5-417c-a512-513f7fadd0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:29.917426 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f89983-10d5-417c-a512-513f7fadd0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:29.917706 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f89983-10d5-417c-a512-513f7fadd0f1 None admin] PATH_INFO: `/v3/projects/7a419183b022401da7b65185607c5bc1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:29.918897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f89983-10d5-417c-a512-513f7fadd0f1 None admin] RBAC: Authorizing `identity:get_project(project_id=7a419183b022401da7b65185607c5bc1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:29.920333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f89983-10d5-417c-a512-513f7fadd0f1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:29.922076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 552/1145] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:43:29 2022] GET /identity/v3/projects/7a419183b022401da7b65185607c5bc1 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:29.928212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc9dd7-004f-4950-a311-9b7d02f88aae None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:29.962775 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:30.036091 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc9dd7-004f-4950-a311-9b7d02f88aae None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:30.073177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94016fab-d29e-4289-8d06-0a4289ed4391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:30.073822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94016fab-d29e-4289-8d06-0a4289ed4391 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:30.074106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94016fab-d29e-4289-8d06-0a4289ed4391 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:30.074383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94016fab-d29e-4289-8d06-0a4289ed4391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:30.075209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94016fab-d29e-4289-8d06-0a4289ed4391 None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=7a419183b022401da7b65185607c5bc1, user.id=e5cd3c8a10634d1e98401cfb47a922d1) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:30.075604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94016fab-d29e-4289-8d06-0a4289ed4391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:30.077550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94016fab-d29e-4289-8d06-0a4289ed4391 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:30.083162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 594/1146] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:43:29 2022] GET /identity/v3/role_assignments?user.id=e5cd3c8a10634d1e98401cfb47a922d1&scope.project.id=7a419183b022401da7b65185607c5bc1&role.id=569707378794437b93d168fc8bb9cbc3 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:32.136380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe4015-975c-4845-ab34-74bee1e3b35d None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:32.136945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe4015-975c-4845-ab34-74bee1e3b35d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:32.137075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe4015-975c-4845-ab34-74bee1e3b35d None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:32.138565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 553/1147] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:32.144480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f9350-f218-4335-b556-c40a6e8a561d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:32.145045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f9350-f218-4335-b556-c40a6e8a561d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:32.145137 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f9350-f218-4335-b556-c40a6e8a561d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:32.162862 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7b7f9350-f218-4335-b556-c40a6e8a561d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:32.164107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7b7f9350-f218-4335-b556-c40a6e8a561d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:32.233013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 595/1148] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:32.255884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe4015-975c-4845-ab34-74bee1e3b35d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:32.348499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:32.359100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe4015-975c-4845-ab34-74bee1e3b35d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:32.392314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec3189-301b-4207-8303-9424391bf552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:32.392963 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec3189-301b-4207-8303-9424391bf552 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:32.393248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec3189-301b-4207-8303-9424391bf552 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:32.393524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec3189-301b-4207-8303-9424391bf552 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:32.403528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ec3189-301b-4207-8303-9424391bf552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-20ec3189-301b-4207-8303-9424391bf552 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:43:32.411892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:32.413017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:32.414064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 554/1149] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:43:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:32.420082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f9350-f218-4335-b556-c40a6e8a561d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:32.447112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:32.518248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f9350-f218-4335-b556-c40a6e8a561d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:32.553254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ed6dd-7e3e-443e-b703-68d7c1d9d2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:32.553928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ed6dd-7e3e-443e-b703-68d7c1d9d2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:32.554217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ed6dd-7e3e-443e-b703-68d7c1d9d2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:32.554499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ed6dd-7e3e-443e-b703-68d7c1d9d2a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:32.555263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ed6dd-7e3e-443e-b703-68d7c1d9d2a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:32.555577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ed6dd-7e3e-443e-b703-68d7c1d9d2a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:32.556973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ed6dd-7e3e-443e-b703-68d7c1d9d2a6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:32.565413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 596/1150] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:43:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:32.571244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec3189-301b-4207-8303-9424391bf552 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:32.674382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec3189-301b-4207-8303-9424391bf552 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:32.714526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d4634-ec1a-48ca-9385-d4072793f181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:32.715413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d4634-ec1a-48ca-9385-d4072793f181 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:32.715694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d4634-ec1a-48ca-9385-d4072793f181 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:32.715976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d4634-ec1a-48ca-9385-d4072793f181 None admin] PATH_INFO: `/v3/users/e5cd3c8a10634d1e98401cfb47a922d1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:32.718011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d4634-ec1a-48ca-9385-d4072793f181 None admin] RBAC: Authorizing `identity:get_user(user_id=e5cd3c8a10634d1e98401cfb47a922d1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:32.719464 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d4634-ec1a-48ca-9385-d4072793f181 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:32.721232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 555/1151] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:32 2022] GET /identity/v3/users/e5cd3c8a10634d1e98401cfb47a922d1 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:32.729731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ed6dd-7e3e-443e-b703-68d7c1d9d2a6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:32.824382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:32.835655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ed6dd-7e3e-443e-b703-68d7c1d9d2a6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:32.870367 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5a917-d88c-450b-a1ee-1fea1dc5a1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:32.871262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5a917-d88c-450b-a1ee-1fea1dc5a1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:32.871545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5a917-d88c-450b-a1ee-1fea1dc5a1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:32.871821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5a917-d88c-450b-a1ee-1fea1dc5a1f2 None admin] PATH_INFO: `/v3/projects/7a419183b022401da7b65185607c5bc1` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:32.873634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5a917-d88c-450b-a1ee-1fea1dc5a1f2 None admin] RBAC: Authorizing `identity:get_project(project_id=7a419183b022401da7b65185607c5bc1)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:32.875085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5a917-d88c-450b-a1ee-1fea1dc5a1f2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:32.877027 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 597/1152] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:43:32 2022] GET /identity/v3/projects/7a419183b022401da7b65185607c5bc1 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:32.886445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d4634-ec1a-48ca-9385-d4072793f181 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:32.911359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:32.994656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d4634-ec1a-48ca-9385-d4072793f181 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:33.035235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5bd96-4950-49ab-867f-854f0c04a38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:33.036116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5bd96-4950-49ab-867f-854f0c04a38e None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:33.036395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5bd96-4950-49ab-867f-854f0c04a38e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:33.036666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5bd96-4950-49ab-867f-854f0c04a38e None admin] PATH_INFO: `/v3/projects/7a419183b022401da7b65185607c5bc1/users/e5cd3c8a10634d1e98401cfb47a922d1/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:33.038723 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5bd96-4950-49ab-867f-854f0c04a38e None admin] RBAC: Authorizing `identity:create_grant(project_id=7a419183b022401da7b65185607c5bc1, user_id=e5cd3c8a10634d1e98401cfb47a922d1, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:33.040135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5bd96-4950-49ab-867f-854f0c04a38e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:33.053071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 556/1153] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:43:32 2022] PUT /identity/v3/projects/7a419183b022401da7b65185607c5bc1/users/e5cd3c8a10634d1e98401cfb47a922d1/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:35.093898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0e60b-7e72-4392-9e35-3d7ca5374605 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:35.094454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0e60b-7e72-4392-9e35-3d7ca5374605 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:35.094454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0e60b-7e72-4392-9e35-3d7ca5374605 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:35.095827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 598/1154] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:35.101046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239151d0-443c-46df-ae5c-fdc6b6939ab4 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:35.101482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239151d0-443c-46df-ae5c-fdc6b6939ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:35.101590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239151d0-443c-46df-ae5c-fdc6b6939ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:35.118625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-239151d0-443c-46df-ae5c-fdc6b6939ab4 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:35.119800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-239151d0-443c-46df-ae5c-fdc6b6939ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:35.185135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 557/1155] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:35.209078 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0e60b-7e72-4392-9e35-3d7ca5374605 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:35.313838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0e60b-7e72-4392-9e35-3d7ca5374605 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:35.349070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770e5e3-5ac9-4bff-b97b-d26d4c6f2477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:35.349780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770e5e3-5ac9-4bff-b97b-d26d4c6f2477 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:35.350067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770e5e3-5ac9-4bff-b97b-d26d4c6f2477 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:35.350349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770e5e3-5ac9-4bff-b97b-d26d4c6f2477 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:35.361435 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770e5e3-5ac9-4bff-b97b-d26d4c6f2477 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-b770e5e3-5ac9-4bff-b97b-d26d4c6f2477 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:43:35.369099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:35.370854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:43:35.371907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 599/1156] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:43:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:35.377260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239151d0-443c-46df-ae5c-fdc6b6939ab4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:35.389997 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:35.477928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239151d0-443c-46df-ae5c-fdc6b6939ab4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:35.511611 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7874c-f6cf-4ff9-bcbb-cf54d1299ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:35.512278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7874c-f6cf-4ff9-bcbb-cf54d1299ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:35.512565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7874c-f6cf-4ff9-bcbb-cf54d1299ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:35.512880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f7874c-f6cf-4ff9-bcbb-cf54d1299ed4 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:35.514149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7874c-f6cf-4ff9-bcbb-cf54d1299ed4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:35.514463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7874c-f6cf-4ff9-bcbb-cf54d1299ed4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:35.515850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f7874c-f6cf-4ff9-bcbb-cf54d1299ed4 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:35.524072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 558/1157] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:43:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:35.530615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770e5e3-5ac9-4bff-b97b-d26d4c6f2477 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:35.550039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:35.635033 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770e5e3-5ac9-4bff-b97b-d26d4c6f2477 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:35.677846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1840d77-51f3-4747-a790-48ee73229227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:35.678732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1840d77-51f3-4747-a790-48ee73229227 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:35.679024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1840d77-51f3-4747-a790-48ee73229227 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:35.679302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1840d77-51f3-4747-a790-48ee73229227 None admin] PATH_INFO: `/v3/users/e5cd3c8a10634d1e98401cfb47a922d1` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:35.680538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1840d77-51f3-4747-a790-48ee73229227 None admin] RBAC: Authorizing `identity:get_user(user_id=e5cd3c8a10634d1e98401cfb47a922d1)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:35.683094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1840d77-51f3-4747-a790-48ee73229227 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:35.685071 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 600/1158] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:35 2022] GET /identity/v3/users/e5cd3c8a10634d1e98401cfb47a922d1 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:35.690939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7874c-f6cf-4ff9-bcbb-cf54d1299ed4 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:35.765483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:35.796757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f7874c-f6cf-4ff9-bcbb-cf54d1299ed4 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:35.831248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d65633-19ef-43a5-a877-6ca7ef67db69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:35.831898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d65633-19ef-43a5-a877-6ca7ef67db69 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:35.832198 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d65633-19ef-43a5-a877-6ca7ef67db69 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:35.832468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d65633-19ef-43a5-a877-6ca7ef67db69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:35.899970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d65633-19ef-43a5-a877-6ca7ef67db69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:35.901378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d65633-19ef-43a5-a877-6ca7ef67db69 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:35.971093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:36.003686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 559/1159] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:43:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:43:36.009762 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1840d77-51f3-4747-a790-48ee73229227 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:36.099744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:36.115941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1840d77-51f3-4747-a790-48ee73229227 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:36.151830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee29c854-b776-4d80-8451-b9eb6170bfe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:36.152615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29c854-b776-4d80-8451-b9eb6170bfe0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:36.152938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29c854-b776-4d80-8451-b9eb6170bfe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:36.153226 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee29c854-b776-4d80-8451-b9eb6170bfe0 None admin] PATH_INFO: `/v3/projects/7a419183b022401da7b65185607c5bc1` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:36.155000 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29c854-b776-4d80-8451-b9eb6170bfe0 None admin] RBAC: Authorizing `identity:get_project(project_id=7a419183b022401da7b65185607c5bc1)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:36.156454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee29c854-b776-4d80-8451-b9eb6170bfe0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:36.158261 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 601/1160] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:43:36 2022] GET /identity/v3/projects/7a419183b022401da7b65185607c5bc1 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:36.165066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d65633-19ef-43a5-a877-6ca7ef67db69 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:36.280868 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d65633-19ef-43a5-a877-6ca7ef67db69 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:36.316382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2cb86-ad60-41b7-9957-a50f0eeadc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:36.317058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2cb86-ad60-41b7-9957-a50f0eeadc31 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:36.317344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2cb86-ad60-41b7-9957-a50f0eeadc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:36.317615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2cb86-ad60-41b7-9957-a50f0eeadc31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:36.318442 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2cb86-ad60-41b7-9957-a50f0eeadc31 None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=7a419183b022401da7b65185607c5bc1, user.id=e5cd3c8a10634d1e98401cfb47a922d1) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:36.318760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2cb86-ad60-41b7-9957-a50f0eeadc31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:36.320181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2cb86-ad60-41b7-9957-a50f0eeadc31 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:36.326480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 560/1161] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:43:36 2022] GET /identity/v3/role_assignments?user.id=e5cd3c8a10634d1e98401cfb47a922d1&scope.project.id=7a419183b022401da7b65185607c5bc1&role.id=569707378794437b93d168fc8bb9cbc3 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:38.401463 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059154ac-1aa6-43f7-a0f2-f0fdde381ca5 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.402010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059154ac-1aa6-43f7-a0f2-f0fdde381ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.402010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059154ac-1aa6-43f7-a0f2-f0fdde381ca5 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.403413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 602/1162] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:43:38.409538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131531a-4213-4922-bef3-17032c523e43 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.410036 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131531a-4213-4922-bef3-17032c523e43 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.410162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131531a-4213-4922-bef3-17032c523e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.427193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8131531a-4213-4922-bef3-17032c523e43 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:38.428418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8131531a-4213-4922-bef3-17032c523e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:38.462174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:38.505703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 561/1163] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:38.528698 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059154ac-1aa6-43f7-a0f2-f0fdde381ca5 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:38.632734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059154ac-1aa6-43f7-a0f2-f0fdde381ca5 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:38.666819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d9714-5339-46d3-9a64-e05649e44de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:38.667409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d9714-5339-46d3-9a64-e05649e44de0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.667698 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d9714-5339-46d3-9a64-e05649e44de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.667982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d9714-5339-46d3-9a64-e05649e44de0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.693343 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:38.739480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d9714-5339-46d3-9a64-e05649e44de0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:38.740954 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0d9714-5339-46d3-9a64-e05649e44de0 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:38.845742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 603/1164] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:43:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:43:38.852238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131531a-4213-4922-bef3-17032c523e43 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:38.951562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131531a-4213-4922-bef3-17032c523e43 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:38.984224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583c51a-7465-4756-b453-84649b33ed9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:38.985054 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583c51a-7465-4756-b453-84649b33ed9b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:38.985339 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583c51a-7465-4756-b453-84649b33ed9b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:38.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583c51a-7465-4756-b453-84649b33ed9b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:38.992521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583c51a-7465-4756-b453-84649b33ed9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:38.993999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583c51a-7465-4756-b453-84649b33ed9b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-3583c51a-7465-4756-b453-84649b33ed9b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:39.008559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:39.010323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:39.011469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 562/1165] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 23 11:43:38 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:39.016392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d9714-5339-46d3-9a64-e05649e44de0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:39.082895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:39.127984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d9714-5339-46d3-9a64-e05649e44de0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:39.163477 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d8fee-9081-459c-ad5a-30a42a021d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:39.164147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d8fee-9081-459c-ad5a-30a42a021d8b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:39.164432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d8fee-9081-459c-ad5a-30a42a021d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:39.164736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d8fee-9081-459c-ad5a-30a42a021d8b None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:39.165462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d8fee-9081-459c-ad5a-30a42a021d8b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:39.165773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d8fee-9081-459c-ad5a-30a42a021d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:39.167184 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d8fee-9081-459c-ad5a-30a42a021d8b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:39.175217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 604/1166] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 23 11:43:39 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:41.220955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9c346-2e82-419b-aa22-2f4dce9afd02 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:41.221595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9c346-2e82-419b-aa22-2f4dce9afd02 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:41.221595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9c346-2e82-419b-aa22-2f4dce9afd02 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:41.222897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 563/1167] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:41.228705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471b154-dee9-40a6-826e-063e573b1f5d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:41.229221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471b154-dee9-40a6-826e-063e573b1f5d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:41.229329 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471b154-dee9-40a6-826e-063e573b1f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:41.247788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4471b154-dee9-40a6-826e-063e573b1f5d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:41.249224 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4471b154-dee9-40a6-826e-063e573b1f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:41.326524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 605/1168] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:41.351725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9c346-2e82-419b-aa22-2f4dce9afd02 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:41.384141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:41.475120 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9c346-2e82-419b-aa22-2f4dce9afd02 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:41.514581 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f936e-d4af-45a3-adcb-a6626a25ba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:41.515246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f936e-d4af-45a3-adcb-a6626a25ba5e None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:41.515534 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f936e-d4af-45a3-adcb-a6626a25ba5e None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:41.515846 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f936e-d4af-45a3-adcb-a6626a25ba5e None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:41.516591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f936e-d4af-45a3-adcb-a6626a25ba5e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:41.518016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f936e-d4af-45a3-adcb-a6626a25ba5e None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:41.534188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 564/1169] 10.222.0.67 () {60 vars in 1138 bytes} [Thu Jun 23 11:43:41 2022] POST /identity/v3/domains => generated 267 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:43.642871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d4797-1adc-40cf-a7a2-79eeb2db8cb2 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:43.643521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d4797-1adc-40cf-a7a2-79eeb2db8cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:43.643521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d4797-1adc-40cf-a7a2-79eeb2db8cb2 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:43.645078 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 606/1170] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:43.651512 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459239db-38fd-4cc7-b01d-e741547386f7 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:43.652058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459239db-38fd-4cc7-b01d-e741547386f7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:43.652058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459239db-38fd-4cc7-b01d-e741547386f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:43.672279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-459239db-38fd-4cc7-b01d-e741547386f7 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:43.673554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-459239db-38fd-4cc7-b01d-e741547386f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:43.747286 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 565/1171] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:43.771728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d4797-1adc-40cf-a7a2-79eeb2db8cb2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:43.827328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:43.881839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d4797-1adc-40cf-a7a2-79eeb2db8cb2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:43.931469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d9de4-c696-4101-ad90-95aab898d0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:43.932485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d9de4-c696-4101-ad90-95aab898d0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:43.932937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d9de4-c696-4101-ad90-95aab898d0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:43.933252 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d9de4-c696-4101-ad90-95aab898d0f3 None admin] PATH_INFO: `/v3/domains/9798dcbfffa447e2b14facf8b57a96ef` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:43.945857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d9de4-c696-4101-ad90-95aab898d0f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9798dcbfffa447e2b14facf8b57a96ef)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:43.947334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8d9de4-c696-4101-ad90-95aab898d0f3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:43.949125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 607/1172] 10.222.0.67 () {56 vars in 1183 bytes} [Thu Jun 23 11:43:43 2022] GET /identity/v3/domains/9798dcbfffa447e2b14facf8b57a96ef => generated 267 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:43.955401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459239db-38fd-4cc7-b01d-e741547386f7 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:43.973597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:44.063180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459239db-38fd-4cc7-b01d-e741547386f7 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:44.102758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edb0270-1f21-46fc-bd9e-27092c6f41c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:44.103344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb0270-1f21-46fc-bd9e-27092c6f41c8 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:44.103626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb0270-1f21-46fc-bd9e-27092c6f41c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:44.103901 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edb0270-1f21-46fc-bd9e-27092c6f41c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:44.104623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb0270-1f21-46fc-bd9e-27092c6f41c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:44.106111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edb0270-1f21-46fc-bd9e-27092c6f41c8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:44.134169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 566/1173] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:43:43 2022] POST /identity/v3/projects => generated 361 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:46.287700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d03d1-46a3-4f4c-8095-99c10335b7a4 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:46.288294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d03d1-46a3-4f4c-8095-99c10335b7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:46.288294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d03d1-46a3-4f4c-8095-99c10335b7a4 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:46.289717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 608/1174] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:46.299561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca38e8-e1d3-439f-bc78-5ce07d64cd9f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:46.300119 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca38e8-e1d3-439f-bc78-5ce07d64cd9f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:46.300612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca38e8-e1d3-439f-bc78-5ce07d64cd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:46.318455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-26ca38e8-e1d3-439f-bc78-5ce07d64cd9f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:46.319687 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-26ca38e8-e1d3-439f-bc78-5ce07d64cd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:46.388990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:46.397401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 567/1175] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:46.423798 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d03d1-46a3-4f4c-8095-99c10335b7a4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:46.571270 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d03d1-46a3-4f4c-8095-99c10335b7a4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:46.582529 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:46.609453 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630bdb-fd58-4119-9016-82a6981a4f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:46.610365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630bdb-fd58-4119-9016-82a6981a4f67 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:46.610692 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630bdb-fd58-4119-9016-82a6981a4f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:46.610992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630bdb-fd58-4119-9016-82a6981a4f67 None admin] PATH_INFO: `/v3/domains/9798dcbfffa447e2b14facf8b57a96ef` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:46.612338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa630bdb-fd58-4119-9016-82a6981a4f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9798dcbfffa447e2b14facf8b57a96ef)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:46.613834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa630bdb-fd58-4119-9016-82a6981a4f67 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:46.615552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 609/1176] 10.222.0.67 () {56 vars in 1183 bytes} [Thu Jun 23 11:43:46 2022] GET /identity/v3/domains/9798dcbfffa447e2b14facf8b57a96ef => generated 267 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:46.621502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca38e8-e1d3-439f-bc78-5ce07d64cd9f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:46.733232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca38e8-e1d3-439f-bc78-5ce07d64cd9f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:46.771232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07d515-6bdd-40b8-a849-6b9d4f5b6c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:46.771915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07d515-6bdd-40b8-a849-6b9d4f5b6c2d None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:46.772201 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07d515-6bdd-40b8-a849-6b9d4f5b6c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:46.772487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07d515-6bdd-40b8-a849-6b9d4f5b6c2d None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:46.773251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07d515-6bdd-40b8-a849-6b9d4f5b6c2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:46.774713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07d515-6bdd-40b8-a849-6b9d4f5b6c2d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:46.797306 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 568/1177] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:43:46 2022] POST /identity/v3/users => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:43:48.955968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc062566-1c33-44f4-af61-1e1d07886d4c None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:48.956517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc062566-1c33-44f4-af61-1e1d07886d4c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:48.956517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc062566-1c33-44f4-af61-1e1d07886d4c None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:48.957968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 610/1178] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:48.963922 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c78c5-7904-4daa-871e-31c66257995f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:48.964503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c78c5-7904-4daa-871e-31c66257995f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:48.964503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c78c5-7904-4daa-871e-31c66257995f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:48.983157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-944c78c5-7904-4daa-871e-31c66257995f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:48.984410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-944c78c5-7904-4daa-871e-31c66257995f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:49.042556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:49.064504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 569/1179] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:49.090180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc062566-1c33-44f4-af61-1e1d07886d4c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:49.224330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc062566-1c33-44f4-af61-1e1d07886d4c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:49.260461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e50cc1-6cf4-4c18-a784-0796526470f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:49.261319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e50cc1-6cf4-4c18-a784-0796526470f0 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:49.261646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e50cc1-6cf4-4c18-a784-0796526470f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:49.261959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e50cc1-6cf4-4c18-a784-0796526470f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:49.274383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e50cc1-6cf4-4c18-a784-0796526470f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-a9e50cc1-6cf4-4c18-a784-0796526470f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:43:49.283002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:49.284543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:43:49.285546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 611/1180] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:43:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:49.291240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c78c5-7904-4daa-871e-31c66257995f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:49.405571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c78c5-7904-4daa-871e-31c66257995f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:49.442424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabafc98-b59b-4373-997e-637a404c4838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:49.443095 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabafc98-b59b-4373-997e-637a404c4838 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:49.443380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabafc98-b59b-4373-997e-637a404c4838 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:49.443656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabafc98-b59b-4373-997e-637a404c4838 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:49.444421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabafc98-b59b-4373-997e-637a404c4838 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:49.444750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabafc98-b59b-4373-997e-637a404c4838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:49.446132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabafc98-b59b-4373-997e-637a404c4838 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:49.457879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 570/1181] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:43:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:49.465310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e50cc1-6cf4-4c18-a784-0796526470f0 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:49.506712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:49.578155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e50cc1-6cf4-4c18-a784-0796526470f0 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:49.614895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6431f-7305-4fdd-b20f-5c3ef0b5aa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:49.615801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6431f-7305-4fdd-b20f-5c3ef0b5aa4c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:49.616089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6431f-7305-4fdd-b20f-5c3ef0b5aa4c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:49.616370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6431f-7305-4fdd-b20f-5c3ef0b5aa4c None admin] PATH_INFO: `/v3/users/01e87f9175f64ec6a91c3cf18bfb60d2` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:49.633366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6431f-7305-4fdd-b20f-5c3ef0b5aa4c None admin] RBAC: Authorizing `identity:get_user(user_id=01e87f9175f64ec6a91c3cf18bfb60d2)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:49.634852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6431f-7305-4fdd-b20f-5c3ef0b5aa4c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:49.636701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 612/1182] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:49 2022] GET /identity/v3/users/01e87f9175f64ec6a91c3cf18bfb60d2 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:49.645622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabafc98-b59b-4373-997e-637a404c4838 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:49.701312 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:49.763177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabafc98-b59b-4373-997e-637a404c4838 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:49.800209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9a631-a175-4d89-8ec7-b47a5a4549e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:49.800897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9a631-a175-4d89-8ec7-b47a5a4549e7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:49.801188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9a631-a175-4d89-8ec7-b47a5a4549e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:49.801471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9a631-a175-4d89-8ec7-b47a5a4549e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:49.872900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9a631-a175-4d89-8ec7-b47a5a4549e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:49.874367 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a9a631-a175-4d89-8ec7-b47a5a4549e7 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:49.921750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:49.980751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 571/1183] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:43:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:43:49.987487 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6431f-7305-4fdd-b20f-5c3ef0b5aa4c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:50.092098 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:50.100088 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6431f-7305-4fdd-b20f-5c3ef0b5aa4c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:50.138309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc3f53-b0e0-4c4c-9da5-fa2f9d929b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:50.139681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc3f53-b0e0-4c4c-9da5-fa2f9d929b53 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:50.139965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc3f53-b0e0-4c4c-9da5-fa2f9d929b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:50.140243 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc3f53-b0e0-4c4c-9da5-fa2f9d929b53 None admin] PATH_INFO: `/v3/projects/10ccedaa931d44948495c3358a9f0c66` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:50.141489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc3f53-b0e0-4c4c-9da5-fa2f9d929b53 None admin] RBAC: Authorizing `identity:get_project(project_id=10ccedaa931d44948495c3358a9f0c66)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:50.142915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc3f53-b0e0-4c4c-9da5-fa2f9d929b53 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:50.144637 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 613/1184] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:43:49 2022] GET /identity/v3/projects/10ccedaa931d44948495c3358a9f0c66 => generated 361 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:50.151172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9a631-a175-4d89-8ec7-b47a5a4549e7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:50.267424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9a631-a175-4d89-8ec7-b47a5a4549e7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:50.306391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed7b2a1-32a7-4056-9a4f-f89158ab5b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:50.307054 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7b2a1-32a7-4056-9a4f-f89158ab5b8f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:50.307344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7b2a1-32a7-4056-9a4f-f89158ab5b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:50.307625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed7b2a1-32a7-4056-9a4f-f89158ab5b8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:50.308460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7b2a1-32a7-4056-9a4f-f89158ab5b8f None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=10ccedaa931d44948495c3358a9f0c66, user.id=01e87f9175f64ec6a91c3cf18bfb60d2) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:50.308811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7b2a1-32a7-4056-9a4f-f89158ab5b8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:50.310760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed7b2a1-32a7-4056-9a4f-f89158ab5b8f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:50.316921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 572/1185] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:43:50 2022] GET /identity/v3/role_assignments?user.id=01e87f9175f64ec6a91c3cf18bfb60d2&scope.project.id=10ccedaa931d44948495c3358a9f0c66&role.id=569707378794437b93d168fc8bb9cbc3 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:52.423912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18f0e-4875-4fb9-b217-f78c38b47a6d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:52.424516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18f0e-4875-4fb9-b217-f78c38b47a6d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:52.424516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18f0e-4875-4fb9-b217-f78c38b47a6d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:52.425921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 614/1186] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:52.432458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e72e06-b489-4427-b3ed-e00cf525d5ed None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:52.433000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e72e06-b489-4427-b3ed-e00cf525d5ed None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:52.433188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e72e06-b489-4427-b3ed-e00cf525d5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:52.450306 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-43e72e06-b489-4427-b3ed-e00cf525d5ed None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:52.452042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-43e72e06-b489-4427-b3ed-e00cf525d5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:52.454964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:52.518897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 573/1187] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:52.542190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18f0e-4875-4fb9-b217-f78c38b47a6d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:52.642810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18f0e-4875-4fb9-b217-f78c38b47a6d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:52.676442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ac9c8-cbad-421c-81d9-25ac10bb4da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:52.677194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ac9c8-cbad-421c-81d9-25ac10bb4da9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:52.677478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ac9c8-cbad-421c-81d9-25ac10bb4da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:52.677751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ac9c8-cbad-421c-81d9-25ac10bb4da9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:52.688939 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36ac9c8-cbad-421c-81d9-25ac10bb4da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e36ac9c8-cbad-421c-81d9-25ac10bb4da9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:43:52.696591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:52.697749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:43:52.698765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 615/1188] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:43:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:52.704766 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e72e06-b489-4427-b3ed-e00cf525d5ed None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:52.810833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e72e06-b489-4427-b3ed-e00cf525d5ed None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:52.845741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39800204-ee48-40d5-9fea-28b787a425c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:52.846414 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39800204-ee48-40d5-9fea-28b787a425c6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:52.846704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39800204-ee48-40d5-9fea-28b787a425c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:52.846981 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39800204-ee48-40d5-9fea-28b787a425c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:52.847739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39800204-ee48-40d5-9fea-28b787a425c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:52.848050 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39800204-ee48-40d5-9fea-28b787a425c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:52.849472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39800204-ee48-40d5-9fea-28b787a425c6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:52.858781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 574/1189] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:43:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:52.864923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ac9c8-cbad-421c-81d9-25ac10bb4da9 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:52.891083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:52.971019 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36ac9c8-cbad-421c-81d9-25ac10bb4da9 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:53.010140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d920e4-7efc-46ed-825f-acaf994af86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:53.011021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d920e4-7efc-46ed-825f-acaf994af86f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:53.011309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d920e4-7efc-46ed-825f-acaf994af86f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:53.011581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d920e4-7efc-46ed-825f-acaf994af86f None admin] PATH_INFO: `/v3/users/01e87f9175f64ec6a91c3cf18bfb60d2` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:53.012917 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d920e4-7efc-46ed-825f-acaf994af86f None admin] RBAC: Authorizing `identity:get_user(user_id=01e87f9175f64ec6a91c3cf18bfb60d2)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:53.014368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d920e4-7efc-46ed-825f-acaf994af86f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:53.016073 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 616/1190] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:52 2022] GET /identity/v3/users/01e87f9175f64ec6a91c3cf18bfb60d2 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:53.022390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39800204-ee48-40d5-9fea-28b787a425c6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:53.025462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:53.131504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39800204-ee48-40d5-9fea-28b787a425c6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:53.168505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867fab7c-258b-4d00-b322-1a3a1cf68eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:53.169989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fab7c-258b-4d00-b322-1a3a1cf68eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:53.170273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fab7c-258b-4d00-b322-1a3a1cf68eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:53.170550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867fab7c-258b-4d00-b322-1a3a1cf68eb8 None admin] PATH_INFO: `/v3/projects/10ccedaa931d44948495c3358a9f0c66` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:53.171838 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fab7c-258b-4d00-b322-1a3a1cf68eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=10ccedaa931d44948495c3358a9f0c66)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:53.173351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867fab7c-258b-4d00-b322-1a3a1cf68eb8 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:53.175149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 575/1191] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:43:53 2022] GET /identity/v3/projects/10ccedaa931d44948495c3358a9f0c66 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:53.181739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d920e4-7efc-46ed-825f-acaf994af86f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:53.286220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:53.297723 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d920e4-7efc-46ed-825f-acaf994af86f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:53.335495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860252f3-eb19-4b0e-bd32-aace004af03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:53.336393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860252f3-eb19-4b0e-bd32-aace004af03e None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:53.336707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860252f3-eb19-4b0e-bd32-aace004af03e None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:53.337185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860252f3-eb19-4b0e-bd32-aace004af03e None admin] PATH_INFO: `/v3/projects/10ccedaa931d44948495c3358a9f0c66/users/01e87f9175f64ec6a91c3cf18bfb60d2/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:53.339240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860252f3-eb19-4b0e-bd32-aace004af03e None admin] RBAC: Authorizing `identity:create_grant(project_id=10ccedaa931d44948495c3358a9f0c66, user_id=01e87f9175f64ec6a91c3cf18bfb60d2, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:53.340696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860252f3-eb19-4b0e-bd32-aace004af03e None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:53.356163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 617/1192] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:43:53 2022] PUT /identity/v3/projects/10ccedaa931d44948495c3358a9f0c66/users/01e87f9175f64ec6a91c3cf18bfb60d2/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:43:55.454289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40c831-8d31-4128-9a9e-51a33af2d08c None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:55.454816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40c831-8d31-4128-9a9e-51a33af2d08c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:55.454922 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f40c831-8d31-4128-9a9e-51a33af2d08c None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:55.456267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 576/1193] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:43:55.461689 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44c754-9ce6-42bc-8ec2-c5a803e1302f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:55.462088 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44c754-9ce6-42bc-8ec2-c5a803e1302f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:55.462244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44c754-9ce6-42bc-8ec2-c5a803e1302f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:55.479169 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bb44c754-9ce6-42bc-8ec2-c5a803e1302f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:43:55.480365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bb44c754-9ce6-42bc-8ec2-c5a803e1302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:43:55.550151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 618/1194] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:43:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:43:55.582484 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40c831-8d31-4128-9a9e-51a33af2d08c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:55.654824 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:55.689244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f40c831-8d31-4128-9a9e-51a33af2d08c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:55.725503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec354a3f-6dd7-44e8-9b43-4216f8ec274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:55.726308 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec354a3f-6dd7-44e8-9b43-4216f8ec274d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:55.726594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec354a3f-6dd7-44e8-9b43-4216f8ec274d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:55.726873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec354a3f-6dd7-44e8-9b43-4216f8ec274d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:55.739074 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec354a3f-6dd7-44e8-9b43-4216f8ec274d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-ec354a3f-6dd7-44e8-9b43-4216f8ec274d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:43:55.746746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:43:55.749423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:43:55.750483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 577/1195] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:43:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:43:55.754522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44c754-9ce6-42bc-8ec2-c5a803e1302f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:55.835756 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:55.855160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44c754-9ce6-42bc-8ec2-c5a803e1302f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:55.890392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5da3de-7ba7-4a67-ae27-73bc025cd9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:55.891051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5da3de-7ba7-4a67-ae27-73bc025cd9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:55.891338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5da3de-7ba7-4a67-ae27-73bc025cd9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:55.891618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5da3de-7ba7-4a67-ae27-73bc025cd9f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:55.892358 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5da3de-7ba7-4a67-ae27-73bc025cd9f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:55.892687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5da3de-7ba7-4a67-ae27-73bc025cd9f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:55.894066 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5da3de-7ba7-4a67-ae27-73bc025cd9f6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:55.902003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 619/1196] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:43:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:55.907785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec354a3f-6dd7-44e8-9b43-4216f8ec274d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:56.003196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec354a3f-6dd7-44e8-9b43-4216f8ec274d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:56.030194 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:56.036296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388a8a2-1575-469d-9298-c3072edba5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:56.037205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388a8a2-1575-469d-9298-c3072edba5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:56.037490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388a8a2-1575-469d-9298-c3072edba5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:56.037760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388a8a2-1575-469d-9298-c3072edba5c5 None admin] PATH_INFO: `/v3/users/01e87f9175f64ec6a91c3cf18bfb60d2` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:56.038954 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388a8a2-1575-469d-9298-c3072edba5c5 None admin] RBAC: Authorizing `identity:get_user(user_id=01e87f9175f64ec6a91c3cf18bfb60d2)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:56.040367 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388a8a2-1575-469d-9298-c3072edba5c5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:56.042006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 578/1197] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 23 11:43:55 2022] GET /identity/v3/users/01e87f9175f64ec6a91c3cf18bfb60d2 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:43:56.048298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5da3de-7ba7-4a67-ae27-73bc025cd9f6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:56.147655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5da3de-7ba7-4a67-ae27-73bc025cd9f6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:56.183569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c184c-27ea-4838-82e9-f8da38e7cbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:56.184041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c184c-27ea-4838-82e9-f8da38e7cbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:56.184325 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c184c-27ea-4838-82e9-f8da38e7cbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:56.185188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c184c-27ea-4838-82e9-f8da38e7cbd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:56.198254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:56.257172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c184c-27ea-4838-82e9-f8da38e7cbd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:56.258537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c184c-27ea-4838-82e9-f8da38e7cbd8 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:56.363247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 620/1198] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:43:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:43:56.369611 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388a8a2-1575-469d-9298-c3072edba5c5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:56.482163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388a8a2-1575-469d-9298-c3072edba5c5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:56.516714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9b36c-bf73-42c6-9595-3c4878494a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:56.517505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9b36c-bf73-42c6-9595-3c4878494a13 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:56.517798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9b36c-bf73-42c6-9595-3c4878494a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:56.518086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9b36c-bf73-42c6-9595-3c4878494a13 None admin] PATH_INFO: `/v3/projects/10ccedaa931d44948495c3358a9f0c66` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:56.519305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9b36c-bf73-42c6-9595-3c4878494a13 None admin] RBAC: Authorizing `identity:get_project(project_id=10ccedaa931d44948495c3358a9f0c66)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:56.520769 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9b36c-bf73-42c6-9595-3c4878494a13 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:56.522504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 579/1199] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 23 11:43:56 2022] GET /identity/v3/projects/10ccedaa931d44948495c3358a9f0c66 => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:43:56.529951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c184c-27ea-4838-82e9-f8da38e7cbd8 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:43:56.573366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:43:56.630029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c184c-27ea-4838-82e9-f8da38e7cbd8 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:43:56.665228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81faf3f0-8fe2-4a78-8236-79e4c2e6f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:43:56.665896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faf3f0-8fe2-4a78-8236-79e4c2e6f774 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:43:56.666185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faf3f0-8fe2-4a78-8236-79e4c2e6f774 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:43:56.666469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81faf3f0-8fe2-4a78-8236-79e4c2e6f774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:43:56.667304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faf3f0-8fe2-4a78-8236-79e4c2e6f774 None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=10ccedaa931d44948495c3358a9f0c66, user.id=01e87f9175f64ec6a91c3cf18bfb60d2) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:43:56.667624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faf3f0-8fe2-4a78-8236-79e4c2e6f774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:43:56.669708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81faf3f0-8fe2-4a78-8236-79e4c2e6f774 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:43:56.675513 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 621/1200] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:43:56 2022] GET /identity/v3/role_assignments?user.id=01e87f9175f64ec6a91c3cf18bfb60d2&scope.project.id=10ccedaa931d44948495c3358a9f0c66&role.id=569707378794437b93d168fc8bb9cbc3 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:06.511701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=67 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:06.672585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=47 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:52.745120 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75389646-140a-4060-baf7-724bc96338ed None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.745762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75389646-140a-4060-baf7-724bc96338ed None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.745762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75389646-140a-4060-baf7-724bc96338ed None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.747152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 580/1201] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:44:52.753160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aebb8-d960-4faa-aa3a-d02c5a05a92f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:52.753712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aebb8-d960-4faa-aa3a-d02c5a05a92f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:52.753712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aebb8-d960-4faa-aa3a-d02c5a05a92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:52.773415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a09aebb8-d960-4faa-aa3a-d02c5a05a92f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:52.774730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a09aebb8-d960-4faa-aa3a-d02c5a05a92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:52.855916 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 622/1202] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:44:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:44:52.882640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75389646-140a-4060-baf7-724bc96338ed None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.028020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75389646-140a-4060-baf7-724bc96338ed None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.078573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9734d-806b-4524-aa3a-827c6a4e1509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.079918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9734d-806b-4524-aa3a-827c6a4e1509 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.080468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9734d-806b-4524-aa3a-827c6a4e1509 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.080819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9734d-806b-4524-aa3a-827c6a4e1509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.093800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9734d-806b-4524-aa3a-827c6a4e1509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.095545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9734d-806b-4524-aa3a-827c6a4e1509 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-86e9734d-806b-4524-aa3a-827c6a4e1509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:44:53.120350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:44:53.121831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:44:53.123202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:44:53.123202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:44:53.123202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:44:53.123202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:44:53.123202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 581/1203] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:44:53.128506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aebb8-d960-4faa-aa3a-d02c5a05a92f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.267109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aebb8-d960-4faa-aa3a-d02c5a05a92f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.299356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:53.311992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206cb31-87dc-4e0c-af14-c5b451e43edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.311992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206cb31-87dc-4e0c-af14-c5b451e43edf None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.311992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206cb31-87dc-4e0c-af14-c5b451e43edf None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.311992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206cb31-87dc-4e0c-af14-c5b451e43edf None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.313073 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206cb31-87dc-4e0c-af14-c5b451e43edf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:53.313073 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206cb31-87dc-4e0c-af14-c5b451e43edf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.313073 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206cb31-87dc-4e0c-af14-c5b451e43edf None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.326823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 623/1204] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:53.333699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9734d-806b-4524-aa3a-827c6a4e1509 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:53.396735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:53.461278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9734d-806b-4524-aa3a-827c6a4e1509 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:53.513182 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56a90c-3446-4f33-8d47-4b390bbac0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:53.514030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56a90c-3446-4f33-8d47-4b390bbac0cf None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:53.514327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56a90c-3446-4f33-8d47-4b390bbac0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:53.514610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56a90c-3446-4f33-8d47-4b390bbac0cf None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:53.515424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56a90c-3446-4f33-8d47-4b390bbac0cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:53.517172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56a90c-3446-4f33-8d47-4b390bbac0cf None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:53.544855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 582/1205] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 23 11:44:53 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:44:55.990959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1365df-d74c-4449-b7f6-c7c498de34a5 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:55.991565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1365df-d74c-4449-b7f6-c7c498de34a5 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:55.991565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1365df-d74c-4449-b7f6-c7c498de34a5 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:55.993071 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 624/1206] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:44:55.999094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd35419-e31d-4aa2-8ad1-1dd43b06dc8a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:55.999514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd35419-e31d-4aa2-8ad1-1dd43b06dc8a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:55.999717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd35419-e31d-4aa2-8ad1-1dd43b06dc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.018807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bcd35419-e31d-4aa2-8ad1-1dd43b06dc8a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:44:56.020101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bcd35419-e31d-4aa2-8ad1-1dd43b06dc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:44:56.105207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 583/1207] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:44:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:44:56.134188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1365df-d74c-4449-b7f6-c7c498de34a5 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.260081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1365df-d74c-4449-b7f6-c7c498de34a5 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.302918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f7823-a454-4b31-886d-8fab5a23df0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.303876 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7823-a454-4b31-886d-8fab5a23df0d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.304282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7823-a454-4b31-886d-8fab5a23df0d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.304588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f7823-a454-4b31-886d-8fab5a23df0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.318603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f7823-a454-4b31-886d-8fab5a23df0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-da4f7823-a454-4b31-886d-8fab5a23df0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:44:56.328925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:44:56.331129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:44:56.332040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 625/1208] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:44:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:44:56.337138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd35419-e31d-4aa2-8ad1-1dd43b06dc8a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.381000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:56.457342 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd35419-e31d-4aa2-8ad1-1dd43b06dc8a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.498230 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c38d71-b64c-43ec-9a2a-8f3fec433ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.499029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c38d71-b64c-43ec-9a2a-8f3fec433ece None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.499346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c38d71-b64c-43ec-9a2a-8f3fec433ece None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.499628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c38d71-b64c-43ec-9a2a-8f3fec433ece None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.501084 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c38d71-b64c-43ec-9a2a-8f3fec433ece None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:56.501405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c38d71-b64c-43ec-9a2a-8f3fec433ece None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.503092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c38d71-b64c-43ec-9a2a-8f3fec433ece None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.517537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 584/1209] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:44:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:56.518717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f7823-a454-4b31-886d-8fab5a23df0d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.580731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:56.643494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f7823-a454-4b31-886d-8fab5a23df0d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.691818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb5892-8131-4724-8ee2-ae8e027b7098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.693439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb5892-8131-4724-8ee2-ae8e027b7098 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.693439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb5892-8131-4724-8ee2-ae8e027b7098 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.693439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb5892-8131-4724-8ee2-ae8e027b7098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.705363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb5892-8131-4724-8ee2-ae8e027b7098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.706993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb5892-8131-4724-8ee2-ae8e027b7098 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-c9cb5892-8131-4724-8ee2-ae8e027b7098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:44:56.728183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:44:56.733599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:44:56.735683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:44:56.735683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:44:56.735683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:44:56.735683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:44:56.735683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 626/1210] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:44:56.736028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c38d71-b64c-43ec-9a2a-8f3fec433ece None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:56.848066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:56.851054 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c38d71-b64c-43ec-9a2a-8f3fec433ece None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:56.891618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adb2be-2f9f-4a2a-be6e-a4ad4ca2e215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:56.892604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adb2be-2f9f-4a2a-be6e-a4ad4ca2e215 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:56.892922 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adb2be-2f9f-4a2a-be6e-a4ad4ca2e215 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:56.893353 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adb2be-2f9f-4a2a-be6e-a4ad4ca2e215 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:56.894127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adb2be-2f9f-4a2a-be6e-a4ad4ca2e215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:56.894463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adb2be-2f9f-4a2a-be6e-a4ad4ca2e215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:56.896167 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adb2be-2f9f-4a2a-be6e-a4ad4ca2e215 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:56.910966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 585/1211] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:56.917059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb5892-8131-4724-8ee2-ae8e027b7098 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.038826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb5892-8131-4724-8ee2-ae8e027b7098 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.040899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:57.078794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816f9ca-2448-41b9-85cd-6a6f50bb6039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.079693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816f9ca-2448-41b9-85cd-6a6f50bb6039 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.080041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816f9ca-2448-41b9-85cd-6a6f50bb6039 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.080384 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816f9ca-2448-41b9-85cd-6a6f50bb6039 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.089434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816f9ca-2448-41b9-85cd-6a6f50bb6039 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.091209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816f9ca-2448-41b9-85cd-6a6f50bb6039 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8816f9ca-2448-41b9-85cd-6a6f50bb6039 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:44:57.109940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:44:57.113446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:44:57.114677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:44:57.114677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:44:57.114677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:44:57.114677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:44:57.114677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:44:57.114677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:44:57.114677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:44:57.114677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:44:57.114677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:44:57.114677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:44:57.115921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 627/1212] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:44:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:44:57.116994 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adb2be-2f9f-4a2a-be6e-a4ad4ca2e215 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.249710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adb2be-2f9f-4a2a-be6e-a4ad4ca2e215 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.299919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26a2a8-8116-44ce-a22b-5cc3a14498c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.300601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26a2a8-8116-44ce-a22b-5cc3a14498c6 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.300992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26a2a8-8116-44ce-a22b-5cc3a14498c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.301373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26a2a8-8116-44ce-a22b-5cc3a14498c6 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.302281 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26a2a8-8116-44ce-a22b-5cc3a14498c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:57.302598 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26a2a8-8116-44ce-a22b-5cc3a14498c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.304338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb26a2a8-8116-44ce-a22b-5cc3a14498c6 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.330027 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 586/1213] 10.222.0.67 () {56 vars in 1143 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:57.335869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816f9ca-2448-41b9-85cd-6a6f50bb6039 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.489803 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816f9ca-2448-41b9-85cd-6a6f50bb6039 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.526645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b66eb-6454-410b-9496-3ea850d8f16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.527304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b66eb-6454-410b-9496-3ea850d8f16f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.527595 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b66eb-6454-410b-9496-3ea850d8f16f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.527955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b66eb-6454-410b-9496-3ea850d8f16f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.563397 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:57.609464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b66eb-6454-410b-9496-3ea850d8f16f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.611008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b66eb-6454-410b-9496-3ea850d8f16f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.744783 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 628/1214] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:44:57.748871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26a2a8-8116-44ce-a22b-5cc3a14498c6 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:57.771392 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:57.865591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26a2a8-8116-44ce-a22b-5cc3a14498c6 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:57.901820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b10094-a8a4-49e3-81c9-c52a4af06b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:57.902613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b10094-a8a4-49e3-81c9-c52a4af06b77 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:57.902895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b10094-a8a4-49e3-81c9-c52a4af06b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:57.903169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b10094-a8a4-49e3-81c9-c52a4af06b77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:57.915968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b10094-a8a4-49e3-81c9-c52a4af06b77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:57.917494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b10094-a8a4-49e3-81c9-c52a4af06b77 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-66b10094-a8a4-49e3-81c9-c52a4af06b77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:44:57.936295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:44:57.937344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:44:57.938434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:44:57.938434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:44:57.938434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:44:57.938434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:44:57.941019 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 587/1215] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:44:57.943217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b66eb-6454-410b-9496-3ea850d8f16f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:58.010615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:58.055092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b66eb-6454-410b-9496-3ea850d8f16f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:58.098522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b71a3-dfd8-4d8b-8f04-07d4be3fd72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:58.099178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b71a3-dfd8-4d8b-8f04-07d4be3fd72c None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:58.099505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b71a3-dfd8-4d8b-8f04-07d4be3fd72c None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:58.099802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b71a3-dfd8-4d8b-8f04-07d4be3fd72c None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:58.100569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b71a3-dfd8-4d8b-8f04-07d4be3fd72c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:58.101157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b71a3-dfd8-4d8b-8f04-07d4be3fd72c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:58.102536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b71a3-dfd8-4d8b-8f04-07d4be3fd72c None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:58.114805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 629/1216] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:58.120238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b10094-a8a4-49e3-81c9-c52a4af06b77 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:58.201275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:58.245660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b10094-a8a4-49e3-81c9-c52a4af06b77 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:58.288366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4605f-33fd-4e34-a121-0655ec81ed48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:58.289290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4605f-33fd-4e34-a121-0655ec81ed48 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:58.289680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4605f-33fd-4e34-a121-0655ec81ed48 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:58.290060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f4605f-33fd-4e34-a121-0655ec81ed48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:58.297263 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4605f-33fd-4e34-a121-0655ec81ed48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:58.298848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f4605f-33fd-4e34-a121-0655ec81ed48 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-24f4605f-33fd-4e34-a121-0655ec81ed48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:44:58.314829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:44:58.315979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:44:58.316943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:44:58.316943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:44:58.316943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:44:58.316943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:44:58.316943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:44:58.316943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:44:58.316943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:44:58.316943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:44:58.316943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 588/1217] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:44:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:44:58.322524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b71a3-dfd8-4d8b-8f04-07d4be3fd72c None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:58.459278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b71a3-dfd8-4d8b-8f04-07d4be3fd72c None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:58.483670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:58.496489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af86b6d-db99-4e83-8a94-84e8017c74a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:58.497180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af86b6d-db99-4e83-8a94-84e8017c74a9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:58.497466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af86b6d-db99-4e83-8a94-84e8017c74a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:58.497743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af86b6d-db99-4e83-8a94-84e8017c74a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:58.498526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af86b6d-db99-4e83-8a94-84e8017c74a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:58.498840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af86b6d-db99-4e83-8a94-84e8017c74a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:58.500860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af86b6d-db99-4e83-8a94-84e8017c74a9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:58.514186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 630/1218] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:44:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:44:58.521271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4605f-33fd-4e34-a121-0655ec81ed48 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:44:58.639556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f4605f-33fd-4e34-a121-0655ec81ed48 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:44:58.677728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:44:58.684925 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc07cc1-e50f-4baa-b8c6-49a0e7eb8271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:44:58.685765 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc07cc1-e50f-4baa-b8c6-49a0e7eb8271 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:44:58.686120 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc07cc1-e50f-4baa-b8c6-49a0e7eb8271 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:44:58.686461 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc07cc1-e50f-4baa-b8c6-49a0e7eb8271 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:44:58.688005 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc07cc1-e50f-4baa-b8c6-49a0e7eb8271 None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=ee971347d23040d3984f961025e8157e) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:44:58.688407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc07cc1-e50f-4baa-b8c6-49a0e7eb8271 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:44:58.690252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc07cc1-e50f-4baa-b8c6-49a0e7eb8271 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:44:58.697327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 589/1219] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:44:58 2022] GET /identity/v3/role_assignments?user.id=ee971347d23040d3984f961025e8157e&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:01.076324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1af88c-32f6-4af1-a02d-73c36db422b2 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.076948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1af88c-32f6-4af1-a02d-73c36db422b2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.076948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1af88c-32f6-4af1-a02d-73c36db422b2 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.079075 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 631/1220] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:45:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:45:01.086629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93246c9d-767d-48b5-87d8-39a633a87f39 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.086629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93246c9d-767d-48b5-87d8-39a633a87f39 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.086629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93246c9d-767d-48b5-87d8-39a633a87f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.104241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-93246c9d-767d-48b5-87d8-39a633a87f39 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:01.105536 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-93246c9d-767d-48b5-87d8-39a633a87f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:01.178504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 590/1221] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:45:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:45:01.201551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1af88c-32f6-4af1-a02d-73c36db422b2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:01.316744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1af88c-32f6-4af1-a02d-73c36db422b2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.355716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19faacb6-b423-423b-986a-430ce18b8d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.356431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faacb6-b423-423b-986a-430ce18b8d3f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.356896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faacb6-b423-423b-986a-430ce18b8d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.357463 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faacb6-b423-423b-986a-430ce18b8d3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.368402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19faacb6-b423-423b-986a-430ce18b8d3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-19faacb6-b423-423b-986a-430ce18b8d3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:45:01.377646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:01.378710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:01.379639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 632/1222] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:45:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:01.387661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93246c9d-767d-48b5-87d8-39a633a87f39 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:01.511987 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93246c9d-767d-48b5-87d8-39a633a87f39 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.545091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:01.557200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5342b8-4e34-456f-9b93-2eb204abc644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.557871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5342b8-4e34-456f-9b93-2eb204abc644 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.558154 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5342b8-4e34-456f-9b93-2eb204abc644 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.558431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5342b8-4e34-456f-9b93-2eb204abc644 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.559186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5342b8-4e34-456f-9b93-2eb204abc644 None admin] RBAC: Adding query filter params (name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:01.559497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5342b8-4e34-456f-9b93-2eb204abc644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.560912 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5342b8-4e34-456f-9b93-2eb204abc644 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.570909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 591/1223] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:45:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:01.576731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19faacb6-b423-423b-986a-430ce18b8d3f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:01.622403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:01.685655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19faacb6-b423-423b-986a-430ce18b8d3f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.721833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644b421-f21e-4e1f-adf3-82e9636bcf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.722622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644b421-f21e-4e1f-adf3-82e9636bcf23 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.722913 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644b421-f21e-4e1f-adf3-82e9636bcf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.723192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644b421-f21e-4e1f-adf3-82e9636bcf23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.735089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644b421-f21e-4e1f-adf3-82e9636bcf23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.736701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644b421-f21e-4e1f-adf3-82e9636bcf23 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8644b421-f21e-4e1f-adf3-82e9636bcf23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:45:01.756399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:01.757603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:01.763188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:45:01.763188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:45:01.763188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:01.763188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:01.763188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 633/1224] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:45:01.763523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5342b8-4e34-456f-9b93-2eb204abc644 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:01.872242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5342b8-4e34-456f-9b93-2eb204abc644 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:01.907601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76551d0-3d66-4afe-a46a-5fc6193bf802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:01.908186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76551d0-3d66-4afe-a46a-5fc6193bf802 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:01.908471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76551d0-3d66-4afe-a46a-5fc6193bf802 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:01.908780 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76551d0-3d66-4afe-a46a-5fc6193bf802 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:01.909499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76551d0-3d66-4afe-a46a-5fc6193bf802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:01.910078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76551d0-3d66-4afe-a46a-5fc6193bf802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:01.911440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76551d0-3d66-4afe-a46a-5fc6193bf802 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:01.925203 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 592/1225] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:01.930996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644b421-f21e-4e1f-adf3-82e9636bcf23 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:02.033268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:02.042144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644b421-f21e-4e1f-adf3-82e9636bcf23 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:02.077885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa009b-9b17-4904-8a19-c187753d4b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:02.078755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa009b-9b17-4904-8a19-c187753d4b8f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:02.079007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa009b-9b17-4904-8a19-c187753d4b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:02.079286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aa009b-9b17-4904-8a19-c187753d4b8f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:02.088018 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa009b-9b17-4904-8a19-c187753d4b8f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:02.089664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aa009b-9b17-4904-8a19-c187753d4b8f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-73aa009b-9b17-4904-8a19-c187753d4b8f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:45:02.114567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:02.115643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:02.116622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:02.116622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:02.116622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:45:02.116622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:45:02.116622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:45:02.116622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:45:02.116622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:45:02.116622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:45:02.116622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:45:02.116622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:02.116622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 634/1226] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:02.123011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76551d0-3d66-4afe-a46a-5fc6193bf802 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:02.176775 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:02.238730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76551d0-3d66-4afe-a46a-5fc6193bf802 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:02.275469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b66af-b526-4e5a-98ba-e65a74b01743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:02.276202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b66af-b526-4e5a-98ba-e65a74b01743 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:02.276451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b66af-b526-4e5a-98ba-e65a74b01743 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:02.276793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b66af-b526-4e5a-98ba-e65a74b01743 None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:02.277637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b66af-b526-4e5a-98ba-e65a74b01743 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:02.277962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b66af-b526-4e5a-98ba-e65a74b01743 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:02.279404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b66af-b526-4e5a-98ba-e65a74b01743 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:02.300080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 593/1227] 10.222.0.67 () {56 vars in 1143 bytes} [Thu Jun 23 11:45:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:45:02.305751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa009b-9b17-4904-8a19-c187753d4b8f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:02.425642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aa009b-9b17-4904-8a19-c187753d4b8f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:02.466197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd707a60-34be-495c-a14d-de3192d9bab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:02.466902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd707a60-34be-495c-a14d-de3192d9bab6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:02.467188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd707a60-34be-495c-a14d-de3192d9bab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:02.467462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd707a60-34be-495c-a14d-de3192d9bab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:02.479169 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd707a60-34be-495c-a14d-de3192d9bab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:02.480839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd707a60-34be-495c-a14d-de3192d9bab6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-cd707a60-34be-495c-a14d-de3192d9bab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:45:02.499690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:02.500898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:02.501875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:45:02.501875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:45:02.501875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:02.501875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:02.501875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 635/1228] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:02.507035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b66af-b526-4e5a-98ba-e65a74b01743 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:02.616747 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b66af-b526-4e5a-98ba-e65a74b01743 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:02.619051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:02.653498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82712fd-af70-470e-8a7d-240431b48fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:02.653934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82712fd-af70-470e-8a7d-240431b48fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:02.654170 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82712fd-af70-470e-8a7d-240431b48fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:02.654489 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82712fd-af70-470e-8a7d-240431b48fe9 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:02.655230 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82712fd-af70-470e-8a7d-240431b48fe9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:02.655547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82712fd-af70-470e-8a7d-240431b48fe9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:02.656984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82712fd-af70-470e-8a7d-240431b48fe9 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:02.669693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 594/1229] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:45:02.675969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd707a60-34be-495c-a14d-de3192d9bab6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:02.688200 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:02.792582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd707a60-34be-495c-a14d-de3192d9bab6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:02.831217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cd257-53a5-4c7e-af9f-9d77be476df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:02.832127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cd257-53a5-4c7e-af9f-9d77be476df6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:02.832488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cd257-53a5-4c7e-af9f-9d77be476df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:02.832872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cd257-53a5-4c7e-af9f-9d77be476df6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:02.839788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cd257-53a5-4c7e-af9f-9d77be476df6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:02.841357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cd257-53a5-4c7e-af9f-9d77be476df6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-a21cd257-53a5-4c7e-af9f-9d77be476df6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:45:02.856338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:02.857391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:02.858945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:45:02.858945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:45:02.858945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:45:02.858945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:45:02.858945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:45:02.858945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:45:02.858945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:45:02.858945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:02.858945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 636/1230] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:45:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:02.863673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82712fd-af70-470e-8a7d-240431b48fe9 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:02.985890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82712fd-af70-470e-8a7d-240431b48fe9 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:03.024651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81136d18-3fd5-4d2f-9cb1-8a61378393f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:03.025374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81136d18-3fd5-4d2f-9cb1-8a61378393f2 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:03.025691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81136d18-3fd5-4d2f-9cb1-8a61378393f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:03.025951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81136d18-3fd5-4d2f-9cb1-8a61378393f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.026850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81136d18-3fd5-4d2f-9cb1-8a61378393f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:03.027172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81136d18-3fd5-4d2f-9cb1-8a61378393f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:03.028742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81136d18-3fd5-4d2f-9cb1-8a61378393f2 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:03.042052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 595/1231] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:45:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:45:03.048278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cd257-53a5-4c7e-af9f-9d77be476df6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:03.118159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:03.155617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cd257-53a5-4c7e-af9f-9d77be476df6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:03.200611 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5143ca1d-c23d-40b3-8fbe-dffe5ea4fa08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:03.201590 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143ca1d-c23d-40b3-8fbe-dffe5ea4fa08 None admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:03.201895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143ca1d-c23d-40b3-8fbe-dffe5ea4fa08 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:03.202222 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5143ca1d-c23d-40b3-8fbe-dffe5ea4fa08 None admin] PATH_INFO: `/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/ee971347d23040d3984f961025e8157e/roles/ff64135ecc4c47ae978c14600bf31853` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:03.217611 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143ca1d-c23d-40b3-8fbe-dffe5ea4fa08 None admin] RBAC: Authorizing `identity:create_grant(project_id=19c2f661cc934aa7869e9b3d4efa875e, user_id=ee971347d23040d3984f961025e8157e, role_id=ff64135ecc4c47ae978c14600bf31853)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:03.219254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5143ca1d-c23d-40b3-8fbe-dffe5ea4fa08 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:03.231271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 637/1232] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:45:03 2022] PUT /identity/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/ee971347d23040d3984f961025e8157e/roles/ff64135ecc4c47ae978c14600bf31853 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:45:05.328720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba5416-ea9f-48ac-8b18-a842f459e9c2 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.329770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba5416-ea9f-48ac-8b18-a842f459e9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.329889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba5416-ea9f-48ac-8b18-a842f459e9c2 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.331343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 596/1233] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:45:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:45:05.338602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594281f5-69c5-4af0-a828-b4bf053829a1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.339103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594281f5-69c5-4af0-a828-b4bf053829a1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.339266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594281f5-69c5-4af0-a828-b4bf053829a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.357209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-594281f5-69c5-4af0-a828-b4bf053829a1 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:05.358649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-594281f5-69c5-4af0-a828-b4bf053829a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:05.439221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 638/1234] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:45:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:45:05.462828 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba5416-ea9f-48ac-8b18-a842f459e9c2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.488835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:05.569278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba5416-ea9f-48ac-8b18-a842f459e9c2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:05.603999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65223cf3-35e9-414a-a6bf-f56b689e2840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:05.604791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65223cf3-35e9-414a-a6bf-f56b689e2840 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.605058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65223cf3-35e9-414a-a6bf-f56b689e2840 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.605335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65223cf3-35e9-414a-a6bf-f56b689e2840 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.615862 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65223cf3-35e9-414a-a6bf-f56b689e2840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-65223cf3-35e9-414a-a6bf-f56b689e2840 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:45:05.623476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:05.625214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:05.630950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 597/1235] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 23 11:45:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:05.631650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594281f5-69c5-4af0-a828-b4bf053829a1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.680940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:05.738947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594281f5-69c5-4af0-a828-b4bf053829a1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:05.776184 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44720c78-fa9a-406c-b887-a8dcc2d69c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:05.776932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44720c78-fa9a-406c-b887-a8dcc2d69c75 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.777251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44720c78-fa9a-406c-b887-a8dcc2d69c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.777534 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44720c78-fa9a-406c-b887-a8dcc2d69c75 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.778382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44720c78-fa9a-406c-b887-a8dcc2d69c75 None admin] RBAC: Adding query filter params (name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:05.778793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44720c78-fa9a-406c-b887-a8dcc2d69c75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:05.780197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44720c78-fa9a-406c-b887-a8dcc2d69c75 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:05.789906 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 639/1236] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 23 11:45:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:45:05.796021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65223cf3-35e9-414a-a6bf-f56b689e2840 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:05.890797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:05.911064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65223cf3-35e9-414a-a6bf-f56b689e2840 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:05.949303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964556f-b699-4e69-a136-05ec08c1632f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:05.950101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964556f-b699-4e69-a136-05ec08c1632f None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:05.950429 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964556f-b699-4e69-a136-05ec08c1632f None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:05.950672 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964556f-b699-4e69-a136-05ec08c1632f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:05.962134 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964556f-b699-4e69-a136-05ec08c1632f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:05.963872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964556f-b699-4e69-a136-05ec08c1632f None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-c964556f-b699-4e69-a136-05ec08c1632f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:45:05.984626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:05.985617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:05.986655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:45:05.986655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:45:05.986655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:05.986655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:05.986655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 598/1237] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:05.991833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44720c78-fa9a-406c-b887-a8dcc2d69c75 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:06.103623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44720c78-fa9a-406c-b887-a8dcc2d69c75 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:06.114289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:06.145832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c9798-ffd1-4463-8a56-933cba9752f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:06.146460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c9798-ffd1-4463-8a56-933cba9752f1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:06.146768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c9798-ffd1-4463-8a56-933cba9752f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:06.147048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c9798-ffd1-4463-8a56-933cba9752f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:06.147821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c9798-ffd1-4463-8a56-933cba9752f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:06.148141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c9798-ffd1-4463-8a56-933cba9752f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:06.149554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c9798-ffd1-4463-8a56-933cba9752f1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:06.163594 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 640/1238] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:06.169590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964556f-b699-4e69-a136-05ec08c1632f None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:06.281697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964556f-b699-4e69-a136-05ec08c1632f None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:06.329941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e1d2b-6548-44c1-9a7c-813984d35047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:06.330810 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e1d2b-6548-44c1-9a7c-813984d35047 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:06.331093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e1d2b-6548-44c1-9a7c-813984d35047 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:06.331368 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383e1d2b-6548-44c1-9a7c-813984d35047 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:06.335757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:06.342116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e1d2b-6548-44c1-9a7c-813984d35047 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:06.344976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383e1d2b-6548-44c1-9a7c-813984d35047 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-383e1d2b-6548-44c1-9a7c-813984d35047 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:45:06.363715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:06.364764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:06.365793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:06.365793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:06.365793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:45:06.365793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:45:06.365793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:45:06.365793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:45:06.365793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:45:06.365793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:45:06.365793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:45:06.365793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:06.365793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 599/1239] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:06.371039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c9798-ffd1-4463-8a56-933cba9752f1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:06.501009 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c9798-ffd1-4463-8a56-933cba9752f1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:06.538856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e65e7-ac27-42eb-840b-a50b2e66921f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:06.539537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e65e7-ac27-42eb-840b-a50b2e66921f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:06.539828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e65e7-ac27-42eb-840b-a50b2e66921f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:06.540125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e65e7-ac27-42eb-840b-a50b2e66921f None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:06.541007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e65e7-ac27-42eb-840b-a50b2e66921f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:06.541328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e65e7-ac27-42eb-840b-a50b2e66921f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:06.542765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e65e7-ac27-42eb-840b-a50b2e66921f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:06.567775 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 641/1240] 10.222.0.67 () {56 vars in 1143 bytes} [Thu Jun 23 11:45:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:45:06.568874 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e1d2b-6548-44c1-9a7c-813984d35047 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:06.685267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383e1d2b-6548-44c1-9a7c-813984d35047 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:06.725414 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cd129-cb88-42c1-bfcd-bd62a9a14c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:06.726151 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cd129-cb88-42c1-bfcd-bd62a9a14c2d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:06.726480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cd129-cb88-42c1-bfcd-bd62a9a14c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:06.726797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cd129-cb88-42c1-bfcd-bd62a9a14c2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:06.798638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:06.807116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5cd129-cb88-42c1-bfcd-bd62a9a14c2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:06.809028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5cd129-cb88-42c1-bfcd-bd62a9a14c2d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:06.922415 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 600/1241] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:45:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 11:45:06.928446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e65e7-ac27-42eb-840b-a50b2e66921f None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:06.958666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:07.035594 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e65e7-ac27-42eb-840b-a50b2e66921f None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:07.076068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6038e19-edb8-4dc5-8c03-8129fa71ab05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:07.076944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6038e19-edb8-4dc5-8c03-8129fa71ab05 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:07.077354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6038e19-edb8-4dc5-8c03-8129fa71ab05 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:07.077776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6038e19-edb8-4dc5-8c03-8129fa71ab05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:07.090488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6038e19-edb8-4dc5-8c03-8129fa71ab05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:07.093126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6038e19-edb8-4dc5-8c03-8129fa71ab05 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e6038e19-edb8-4dc5-8c03-8129fa71ab05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:45:07.113047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:07.114605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:07.115554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:45:07.115554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:45:07.115554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:07.115554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:07.115554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 642/1242] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:07.120226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cd129-cb88-42c1-bfcd-bd62a9a14c2d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:07.231871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:07.240806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cd129-cb88-42c1-bfcd-bd62a9a14c2d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:07.280006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2152e0-52b8-446c-bd03-46a1ab2cff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:07.280753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2152e0-52b8-446c-bd03-46a1ab2cff82 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:07.281613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2152e0-52b8-446c-bd03-46a1ab2cff82 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:07.281897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2152e0-52b8-446c-bd03-46a1ab2cff82 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:07.282732 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2152e0-52b8-446c-bd03-46a1ab2cff82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:07.282971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2152e0-52b8-446c-bd03-46a1ab2cff82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:07.284410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2152e0-52b8-446c-bd03-46a1ab2cff82 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:07.298413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 601/1243] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:07.303976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6038e19-edb8-4dc5-8c03-8129fa71ab05 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:07.395196 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:07.425734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6038e19-edb8-4dc5-8c03-8129fa71ab05 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:07.462591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07309dee-6412-4c24-aff7-723fb363cdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:07.463876 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07309dee-6412-4c24-aff7-723fb363cdd3 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:07.464181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07309dee-6412-4c24-aff7-723fb363cdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:07.464564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07309dee-6412-4c24-aff7-723fb363cdd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:07.472848 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07309dee-6412-4c24-aff7-723fb363cdd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:07.474550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07309dee-6412-4c24-aff7-723fb363cdd3 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-07309dee-6412-4c24-aff7-723fb363cdd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:45:07.489669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:07.490826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:07.491955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:45:07.491955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:45:07.491955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:45:07.491955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:45:07.491955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:45:07.491955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:45:07.491955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:45:07.491955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:07.491955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 643/1244] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:45:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:07.496914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2152e0-52b8-446c-bd03-46a1ab2cff82 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:07.608340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2152e0-52b8-446c-bd03-46a1ab2cff82 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:07.645515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e061e-2619-4283-bc8a-b58de068cc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:07.646128 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e061e-2619-4283-bc8a-b58de068cc9b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:07.646358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e061e-2619-4283-bc8a-b58de068cc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:07.646684 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e061e-2619-4283-bc8a-b58de068cc9b None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:07.647504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e061e-2619-4283-bc8a-b58de068cc9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:07.647818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e061e-2619-4283-bc8a-b58de068cc9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:07.649764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e061e-2619-4283-bc8a-b58de068cc9b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:07.672971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 602/1245] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:45:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:07.674449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07309dee-6412-4c24-aff7-723fb363cdd3 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:07.784133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07309dee-6412-4c24-aff7-723fb363cdd3 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:07.822743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96586a-b4a8-4cb6-b39a-62c9df59b57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:07.823405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96586a-b4a8-4cb6-b39a-62c9df59b57f None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:07.823709 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96586a-b4a8-4cb6-b39a-62c9df59b57f None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:07.824067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96586a-b4a8-4cb6-b39a-62c9df59b57f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:07.824993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96586a-b4a8-4cb6-b39a-62c9df59b57f None admin] RBAC: Adding query filter params (role.id=ff64135ecc4c47ae978c14600bf31853, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=ee971347d23040d3984f961025e8157e) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:07.825412 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96586a-b4a8-4cb6-b39a-62c9df59b57f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:07.826909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96586a-b4a8-4cb6-b39a-62c9df59b57f None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:07.833975 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 644/1246] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:45:07 2022] GET /identity/v3/role_assignments?user.id=ee971347d23040d3984f961025e8157e&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=ff64135ecc4c47ae978c14600bf31853 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:45:10.009601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e95686-c465-4b91-9efb-4edb3c4ff4e6 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:10.010264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e95686-c465-4b91-9efb-4edb3c4ff4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:10.010398 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e95686-c465-4b91-9efb-4edb3c4ff4e6 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:10.012061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 603/1247] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:45:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:45:10.018858 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09edf8-9121-4462-80ff-825d78283ee6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:10.019383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09edf8-9121-4462-80ff-825d78283ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:10.019490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09edf8-9121-4462-80ff-825d78283ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:10.024769 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:10.038213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2a09edf8-9121-4462-80ff-825d78283ee6 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:10.040014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2a09edf8-9121-4462-80ff-825d78283ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:10.139455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 645/1248] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:45:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:45:10.169503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e95686-c465-4b91-9efb-4edb3c4ff4e6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:10.197383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:10.299653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e95686-c465-4b91-9efb-4edb3c4ff4e6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:10.340326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0415057-7715-478e-b50b-96ceb187a97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:10.341163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0415057-7715-478e-b50b-96ceb187a97b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:10.341463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0415057-7715-478e-b50b-96ceb187a97b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:10.341723 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0415057-7715-478e-b50b-96ceb187a97b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:10.355445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0415057-7715-478e-b50b-96ceb187a97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-f0415057-7715-478e-b50b-96ceb187a97b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:45:10.364129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:10.365207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:10.366064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 604/1249] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:45:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:10.371600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09edf8-9121-4462-80ff-825d78283ee6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:10.490934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09edf8-9121-4462-80ff-825d78283ee6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:10.524066 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:10.530162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c9cb4-a65d-40dd-81ab-86d86ea80aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:10.531099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c9cb4-a65d-40dd-81ab-86d86ea80aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:10.531391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c9cb4-a65d-40dd-81ab-86d86ea80aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:10.531689 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61c9cb4-a65d-40dd-81ab-86d86ea80aa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:10.534012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c9cb4-a65d-40dd-81ab-86d86ea80aa7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:10.534333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c9cb4-a65d-40dd-81ab-86d86ea80aa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:10.536090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61c9cb4-a65d-40dd-81ab-86d86ea80aa7 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:10.551724 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 646/1250] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:45:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:10.552952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0415057-7715-478e-b50b-96ceb187a97b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:10.643035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:10.677618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0415057-7715-478e-b50b-96ceb187a97b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:10.718186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08d99a-af41-4705-b7ea-ccf2c11dd624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:10.719478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08d99a-af41-4705-b7ea-ccf2c11dd624 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:10.719954 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08d99a-af41-4705-b7ea-ccf2c11dd624 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:10.720369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08d99a-af41-4705-b7ea-ccf2c11dd624 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:10.738318 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08d99a-af41-4705-b7ea-ccf2c11dd624 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:10.739775 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf08d99a-af41-4705-b7ea-ccf2c11dd624 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-cf08d99a-af41-4705-b7ea-ccf2c11dd624 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:45:10.760982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:10.762176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:10.763436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:45:10.763436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:45:10.763436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:10.763436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:10.763436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 605/1251] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 11:45:10.768482 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c9cb4-a65d-40dd-81ab-86d86ea80aa7 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:10.882127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61c9cb4-a65d-40dd-81ab-86d86ea80aa7 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:10.920539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32733110-82f8-4142-bd17-c3bfb56bbfe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:10.921263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32733110-82f8-4142-bd17-c3bfb56bbfe2 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:10.922155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32733110-82f8-4142-bd17-c3bfb56bbfe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:10.922448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32733110-82f8-4142-bd17-c3bfb56bbfe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:10.923217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32733110-82f8-4142-bd17-c3bfb56bbfe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:10.923534 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32733110-82f8-4142-bd17-c3bfb56bbfe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:10.925200 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32733110-82f8-4142-bd17-c3bfb56bbfe2 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:10.939742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 647/1252] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:10.945290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08d99a-af41-4705-b7ea-ccf2c11dd624 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:11.059496 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf08d99a-af41-4705-b7ea-ccf2c11dd624 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:11.091229 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:11.100922 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db47b33-f87d-41f3-bde1-bca39e724c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:11.101700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db47b33-f87d-41f3-bde1-bca39e724c36 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:11.101987 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db47b33-f87d-41f3-bde1-bca39e724c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:11.102267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db47b33-f87d-41f3-bde1-bca39e724c36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:11.111413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db47b33-f87d-41f3-bde1-bca39e724c36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:11.113222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db47b33-f87d-41f3-bde1-bca39e724c36 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0db47b33-f87d-41f3-bde1-bca39e724c36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:45:11.130271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:11.131500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:11.132949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:11.132949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:11.132949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:45:11.132949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:45:11.132949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:45:11.132949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:45:11.132949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:45:11.132949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:45:11.132949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:45:11.132949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:11.132949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 606/1253] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:11.137887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32733110-82f8-4142-bd17-c3bfb56bbfe2 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:11.206044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:11.257489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32733110-82f8-4142-bd17-c3bfb56bbfe2 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:11.298371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0d7a1-ae64-4c38-9ebc-8be71b600f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:11.299045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0d7a1-ae64-4c38-9ebc-8be71b600f85 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:11.299334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0d7a1-ae64-4c38-9ebc-8be71b600f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:11.299652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea0d7a1-ae64-4c38-9ebc-8be71b600f85 None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:11.300594 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0d7a1-ae64-4c38-9ebc-8be71b600f85 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:11.300996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0d7a1-ae64-4c38-9ebc-8be71b600f85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:11.302685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea0d7a1-ae64-4c38-9ebc-8be71b600f85 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:11.322224 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 648/1254] 10.222.0.67 () {56 vars in 1143 bytes} [Thu Jun 23 11:45:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:11.328554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db47b33-f87d-41f3-bde1-bca39e724c36 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:11.450178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db47b33-f87d-41f3-bde1-bca39e724c36 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:11.492295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbd704-5366-45d3-9d66-8d5dc325f89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:11.493045 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbd704-5366-45d3-9d66-8d5dc325f89b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:11.493333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbd704-5366-45d3-9d66-8d5dc325f89b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:11.493788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edbd704-5366-45d3-9d66-8d5dc325f89b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:11.557494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:11.571499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbd704-5366-45d3-9d66-8d5dc325f89b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:11.573091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edbd704-5366-45d3-9d66-8d5dc325f89b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:11.689116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 607/1255] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:45:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:45:11.696951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0d7a1-ae64-4c38-9ebc-8be71b600f85 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:11.818870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea0d7a1-ae64-4c38-9ebc-8be71b600f85 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:11.836159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:11.863498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86a533-b013-4b03-89ae-79a988f5fc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:11.864589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86a533-b013-4b03-89ae-79a988f5fc71 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:11.864912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86a533-b013-4b03-89ae-79a988f5fc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:11.865320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86a533-b013-4b03-89ae-79a988f5fc71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:11.879361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86a533-b013-4b03-89ae-79a988f5fc71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:11.880840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86a533-b013-4b03-89ae-79a988f5fc71 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-1a86a533-b013-4b03-89ae-79a988f5fc71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:45:11.900920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:11.902055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:11.903289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:45:11.903289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:45:11.903289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:11.903289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:11.903289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 649/1256] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:11.907797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbd704-5366-45d3-9d66-8d5dc325f89b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:11.999069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:12.013842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edbd704-5366-45d3-9d66-8d5dc325f89b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:12.049844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad6687-0ff3-407d-8863-e70a1b776e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:12.050523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad6687-0ff3-407d-8863-e70a1b776e0b None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:12.050812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad6687-0ff3-407d-8863-e70a1b776e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:12.051092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad6687-0ff3-407d-8863-e70a1b776e0b None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:12.051893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad6687-0ff3-407d-8863-e70a1b776e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:12.052207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad6687-0ff3-407d-8863-e70a1b776e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:12.053642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad6687-0ff3-407d-8863-e70a1b776e0b None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:12.066162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 608/1257] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:12.071897 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86a533-b013-4b03-89ae-79a988f5fc71 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:12.179898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86a533-b013-4b03-89ae-79a988f5fc71 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:12.216397 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1f8ac-dc98-48c2-b595-adc0045727f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:12.217376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1f8ac-dc98-48c2-b595-adc0045727f6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:12.217696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1f8ac-dc98-48c2-b595-adc0045727f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:12.217974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c1f8ac-dc98-48c2-b595-adc0045727f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:12.226550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1f8ac-dc98-48c2-b595-adc0045727f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:12.228355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c1f8ac-dc98-48c2-b595-adc0045727f6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-e4c1f8ac-dc98-48c2-b595-adc0045727f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:45:12.243838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:12.245193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:12.246912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:45:12.246912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:45:12.246912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:45:12.246912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:45:12.246912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:45:12.246912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:45:12.246912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:45:12.246912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:12.246912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 650/1258] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:45:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:12.252883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad6687-0ff3-407d-8863-e70a1b776e0b None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:12.361676 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad6687-0ff3-407d-8863-e70a1b776e0b None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:12.396418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34c56b-c172-4773-a669-7b431b743571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:12.397112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34c56b-c172-4773-a669-7b431b743571 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:12.397430 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34c56b-c172-4773-a669-7b431b743571 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:12.397674 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34c56b-c172-4773-a669-7b431b743571 None admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:12.398451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34c56b-c172-4773-a669-7b431b743571 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:12.398771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34c56b-c172-4773-a669-7b431b743571 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:12.400312 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34c56b-c172-4773-a669-7b431b743571 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:12.413225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 609/1259] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:45:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:12.420011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1f8ac-dc98-48c2-b595-adc0045727f6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:12.453100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:12.535994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c1f8ac-dc98-48c2-b595-adc0045727f6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:12.581265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439bdd3-96f3-4512-8b84-f0adb5a9df54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:12.581929 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439bdd3-96f3-4512-8b84-f0adb5a9df54 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:12.582392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439bdd3-96f3-4512-8b84-f0adb5a9df54 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:12.582687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439bdd3-96f3-4512-8b84-f0adb5a9df54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:12.583628 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439bdd3-96f3-4512-8b84-f0adb5a9df54 None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=ee971347d23040d3984f961025e8157e) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:12.583945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439bdd3-96f3-4512-8b84-f0adb5a9df54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:12.585437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439bdd3-96f3-4512-8b84-f0adb5a9df54 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:12.593919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 651/1260] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:45:12 2022] GET /identity/v3/role_assignments?user.id=ee971347d23040d3984f961025e8157e&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=569707378794437b93d168fc8bb9cbc3 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:14.656052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97eb87-5088-4187-8254-166d140b9cfb None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:14.656582 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97eb87-5088-4187-8254-166d140b9cfb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:14.656702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97eb87-5088-4187-8254-166d140b9cfb None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:14.658036 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 610/1261] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:45:14.663846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbedff3-e5c3-4c2a-bd3e-772a99e40eb6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:14.664282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbedff3-e5c3-4c2a-bd3e-772a99e40eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:14.664412 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbedff3-e5c3-4c2a-bd3e-772a99e40eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:14.681222 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2cbedff3-e5c3-4c2a-bd3e-772a99e40eb6 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:14.682407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2cbedff3-e5c3-4c2a-bd3e-772a99e40eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:14.747141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 652/1262] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:45:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:45:14.770709 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97eb87-5088-4187-8254-166d140b9cfb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:14.781941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:14.870204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97eb87-5088-4187-8254-166d140b9cfb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:14.901812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5090fec-d53c-46e4-b2ca-aeb90f3895b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:14.902583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5090fec-d53c-46e4-b2ca-aeb90f3895b7 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:14.902936 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5090fec-d53c-46e4-b2ca-aeb90f3895b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:14.903214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5090fec-d53c-46e4-b2ca-aeb90f3895b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:14.913682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5090fec-d53c-46e4-b2ca-aeb90f3895b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-a5090fec-d53c-46e4-b2ca-aeb90f3895b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:45:14.921098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:14.922804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:14.923735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 611/1263] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:45:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:14.928934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbedff3-e5c3-4c2a-bd3e-772a99e40eb6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:14.946521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:15.024859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbedff3-e5c3-4c2a-bd3e-772a99e40eb6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:15.058334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1808559-6bf9-4d7c-a0a9-bca6ff05aee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:15.058984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1808559-6bf9-4d7c-a0a9-bca6ff05aee9 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.059266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1808559-6bf9-4d7c-a0a9-bca6ff05aee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.059537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1808559-6bf9-4d7c-a0a9-bca6ff05aee9 None admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.060345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1808559-6bf9-4d7c-a0a9-bca6ff05aee9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:15.060657 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1808559-6bf9-4d7c-a0a9-bca6ff05aee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:15.062070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1808559-6bf9-4d7c-a0a9-bca6ff05aee9 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.069830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 653/1264] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:45:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:15.075887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5090fec-d53c-46e4-b2ca-aeb90f3895b7 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:15.141389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:15.174974 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5090fec-d53c-46e4-b2ca-aeb90f3895b7 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:15.207863 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf68d7-e692-434b-80a9-37a712e762d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:15.208649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf68d7-e692-434b-80a9-37a712e762d1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.208970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf68d7-e692-434b-80a9-37a712e762d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.209250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf68d7-e692-434b-80a9-37a712e762d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.220270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf68d7-e692-434b-80a9-37a712e762d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:15.221750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf68d7-e692-434b-80a9-37a712e762d1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-0baf68d7-e692-434b-80a9-37a712e762d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:45:15.238488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:15.239614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:15.240799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:45:15.240799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:45:15.240799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:15.240799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:15.240799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 612/1265] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:15.245577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1808559-6bf9-4d7c-a0a9-bca6ff05aee9 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:15.323759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:15.342999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1808559-6bf9-4d7c-a0a9-bca6ff05aee9 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:15.376285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb837f-9fdb-4c72-8d25-11c5aa052d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:15.376964 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb837f-9fdb-4c72-8d25-11c5aa052d6a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.377251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb837f-9fdb-4c72-8d25-11c5aa052d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.377531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb837f-9fdb-4c72-8d25-11c5aa052d6a None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.378265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb837f-9fdb-4c72-8d25-11c5aa052d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:15.378577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb837f-9fdb-4c72-8d25-11c5aa052d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:15.380699 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb837f-9fdb-4c72-8d25-11c5aa052d6a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.393938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 654/1266] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:15.399800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf68d7-e692-434b-80a9-37a712e762d1 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:15.506419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf68d7-e692-434b-80a9-37a712e762d1 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:15.525763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:15.539918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844dab3-3e30-43a6-843a-f53a40b8d96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:15.540730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844dab3-3e30-43a6-843a-f53a40b8d96d None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.541016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844dab3-3e30-43a6-843a-f53a40b8d96d None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.541292 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e844dab3-3e30-43a6-843a-f53a40b8d96d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.548616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844dab3-3e30-43a6-843a-f53a40b8d96d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:15.550102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e844dab3-3e30-43a6-843a-f53a40b8d96d None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-e844dab3-3e30-43a6-843a-f53a40b8d96d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:45:15.565570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:15.566751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:15.567890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:15.567890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:15.567890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:45:15.567890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:45:15.567890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:45:15.567890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:45:15.567890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:45:15.567890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:45:15.567890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:45:15.567890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:15.567890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 613/1267] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:15.572586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb837f-9fdb-4c72-8d25-11c5aa052d6a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:15.670159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb837f-9fdb-4c72-8d25-11c5aa052d6a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:15.703624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dcdfb-05cc-4385-8e1d-79be0f3e659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:15.704281 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dcdfb-05cc-4385-8e1d-79be0f3e659a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.704567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dcdfb-05cc-4385-8e1d-79be0f3e659a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.704874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72dcdfb-05cc-4385-8e1d-79be0f3e659a None admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.705700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dcdfb-05cc-4385-8e1d-79be0f3e659a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:15.706015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dcdfb-05cc-4385-8e1d-79be0f3e659a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:15.707432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72dcdfb-05cc-4385-8e1d-79be0f3e659a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.715753 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:15.724493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 655/1268] 10.222.0.67 () {56 vars in 1143 bytes} [Thu Jun 23 11:45:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:15.730056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844dab3-3e30-43a6-843a-f53a40b8d96d None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:15.828570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e844dab3-3e30-43a6-843a-f53a40b8d96d None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:15.862478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f9339-589d-41c8-aad0-924f350dc228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:15.863272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f9339-589d-41c8-aad0-924f350dc228 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:15.863559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f9339-589d-41c8-aad0-924f350dc228 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:15.863837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f9339-589d-41c8-aad0-924f350dc228 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:15.875483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f9339-589d-41c8-aad0-924f350dc228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:15.876957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f9339-589d-41c8-aad0-924f350dc228 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-107f9339-589d-41c8-aad0-924f350dc228 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:45:15.894135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:15.895274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:15.896383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:45:15.896383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:45:15.896383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:15.896383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:15.896383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 614/1269] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:15.901393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dcdfb-05cc-4385-8e1d-79be0f3e659a None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:16.000742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72dcdfb-05cc-4385-8e1d-79be0f3e659a None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:16.036471 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c77c0-f26a-4784-92ca-f8b1c59b9610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:16.037153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c77c0-f26a-4784-92ca-f8b1c59b9610 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:16.037441 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c77c0-f26a-4784-92ca-f8b1c59b9610 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:16.037717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90c77c0-f26a-4784-92ca-f8b1c59b9610 None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:16.038433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c77c0-f26a-4784-92ca-f8b1c59b9610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:16.038848 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c77c0-f26a-4784-92ca-f8b1c59b9610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:16.040214 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90c77c0-f26a-4784-92ca-f8b1c59b9610 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:16.051602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 656/1270] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:16.057289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f9339-589d-41c8-aad0-924f350dc228 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:16.088656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:16.159456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f9339-589d-41c8-aad0-924f350dc228 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:16.192500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d9cb8-33d6-46e8-a19a-de4d2c8cc793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:16.193307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d9cb8-33d6-46e8-a19a-de4d2c8cc793 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:16.193599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d9cb8-33d6-46e8-a19a-de4d2c8cc793 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:16.193881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d9cb8-33d6-46e8-a19a-de4d2c8cc793 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:16.200352 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d9cb8-33d6-46e8-a19a-de4d2c8cc793 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:16.201823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d9cb8-33d6-46e8-a19a-de4d2c8cc793 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-331d9cb8-33d6-46e8-a19a-de4d2c8cc793 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:45:16.218818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:16.219941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:16.221249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:45:16.221249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:45:16.221249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:45:16.221249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:45:16.221249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:45:16.221249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:45:16.221249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:45:16.221249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:16.221249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 615/1271] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:45:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:16.225869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c77c0-f26a-4784-92ca-f8b1c59b9610 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:16.286720 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:16.327687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90c77c0-f26a-4784-92ca-f8b1c59b9610 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:16.361046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b57689-5cbd-4120-a4c2-5e940a5b087b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:16.361623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b57689-5cbd-4120-a4c2-5e940a5b087b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:16.361909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b57689-5cbd-4120-a4c2-5e940a5b087b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:16.362188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b57689-5cbd-4120-a4c2-5e940a5b087b None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:16.362972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b57689-5cbd-4120-a4c2-5e940a5b087b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:16.363284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b57689-5cbd-4120-a4c2-5e940a5b087b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:16.364813 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b57689-5cbd-4120-a4c2-5e940a5b087b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:16.377337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 657/1272] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:45:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:16.383675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d9cb8-33d6-46e8-a19a-de4d2c8cc793 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:16.482177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:16.491492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d9cb8-33d6-46e8-a19a-de4d2c8cc793 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:16.523472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd10ee-1351-470c-a67e-462c066da3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:16.524357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd10ee-1351-470c-a67e-462c066da3cd None admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:16.524638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd10ee-1351-470c-a67e-462c066da3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:16.524944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd10ee-1351-470c-a67e-462c066da3cd None admin] PATH_INFO: `/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/ee971347d23040d3984f961025e8157e/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:16.526881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd10ee-1351-470c-a67e-462c066da3cd None admin] RBAC: Authorizing `identity:create_grant(project_id=19c2f661cc934aa7869e9b3d4efa875e, user_id=ee971347d23040d3984f961025e8157e, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:16.528303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd10ee-1351-470c-a67e-462c066da3cd None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:16.541511 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 616/1273] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 23 11:45:16 2022] PUT /identity/v3/projects/19c2f661cc934aa7869e9b3d4efa875e/users/ee971347d23040d3984f961025e8157e/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:45:18.548775 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86e413-f557-4af9-9060-b276d9360057 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:18.549366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86e413-f557-4af9-9060-b276d9360057 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:18.549366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea86e413-f557-4af9-9060-b276d9360057 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:18.550693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 658/1274] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:45:18.555911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e961bc3-1155-46ae-8ad3-942ec4d193c3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:18.556304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e961bc3-1155-46ae-8ad3-942ec4d193c3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:18.556480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e961bc3-1155-46ae-8ad3-942ec4d193c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:18.573293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6e961bc3-1155-46ae-8ad3-942ec4d193c3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:18.574502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6e961bc3-1155-46ae-8ad3-942ec4d193c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:18.638299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 617/1275] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:45:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:45:18.662649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86e413-f557-4af9-9060-b276d9360057 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:18.760419 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea86e413-f557-4af9-9060-b276d9360057 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:18.777971 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:18.796466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337fd60-3d28-4a1a-bf46-f1a36a73ac51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:18.797279 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337fd60-3d28-4a1a-bf46-f1a36a73ac51 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:18.797570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337fd60-3d28-4a1a-bf46-f1a36a73ac51 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:18.797851 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337fd60-3d28-4a1a-bf46-f1a36a73ac51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:18.808204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337fd60-3d28-4a1a-bf46-f1a36a73ac51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-9337fd60-3d28-4a1a-bf46-f1a36a73ac51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 11:45:18.816300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:18.817585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:18.818540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 659/1276] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 23 11:45:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:18.823505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e961bc3-1155-46ae-8ad3-942ec4d193c3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:18.902933 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:18.921991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e961bc3-1155-46ae-8ad3-942ec4d193c3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:18.955269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc3f7a-c73a-4355-805d-72a733c40cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:18.955932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc3f7a-c73a-4355-805d-72a733c40cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:18.956217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc3f7a-c73a-4355-805d-72a733c40cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:18.956496 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc3f7a-c73a-4355-805d-72a733c40cf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:18.957271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc3f7a-c73a-4355-805d-72a733c40cf5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:18.957577 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc3f7a-c73a-4355-805d-72a733c40cf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:18.958949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc3f7a-c73a-4355-805d-72a733c40cf5 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:18.966857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 618/1277] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 23 11:45:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:18.972468 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337fd60-3d28-4a1a-bf46-f1a36a73ac51 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:19.068024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337fd60-3d28-4a1a-bf46-f1a36a73ac51 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:19.099396 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d23aed-4ff3-44ae-b610-c7b2444ad85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:19.100167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d23aed-4ff3-44ae-b610-c7b2444ad85d None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:19.100452 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d23aed-4ff3-44ae-b610-c7b2444ad85d None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:19.100757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d23aed-4ff3-44ae-b610-c7b2444ad85d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:19.111419 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d23aed-4ff3-44ae-b610-c7b2444ad85d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:19.112873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d23aed-4ff3-44ae-b610-c7b2444ad85d None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-10d23aed-4ff3-44ae-b610-c7b2444ad85d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:45:19.130275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:19.131368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:19.132349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:45:19.132349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:45:19.132349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:19.132349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:19.132349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 660/1278] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:19.137114 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc3f7a-c73a-4355-805d-72a733c40cf5 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:19.233256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc3f7a-c73a-4355-805d-72a733c40cf5 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:19.266249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1516a-f4ad-4fe8-bca9-bf26319e24d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:19.266898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1516a-f4ad-4fe8-bca9-bf26319e24d3 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:19.267185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1516a-f4ad-4fe8-bca9-bf26319e24d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:19.267462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1516a-f4ad-4fe8-bca9-bf26319e24d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:19.268181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1516a-f4ad-4fe8-bca9-bf26319e24d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:19.268489 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1516a-f4ad-4fe8-bca9-bf26319e24d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:19.269867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1516a-f4ad-4fe8-bca9-bf26319e24d3 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:19.281754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 619/1279] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:19.287630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d23aed-4ff3-44ae-b610-c7b2444ad85d None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:19.318174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:19.390728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d23aed-4ff3-44ae-b610-c7b2444ad85d None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:19.432011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792a16a-21e0-4f91-84a0-9538cd9046b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:19.432925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792a16a-21e0-4f91-84a0-9538cd9046b1 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:19.433439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792a16a-21e0-4f91-84a0-9538cd9046b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:19.433565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792a16a-21e0-4f91-84a0-9538cd9046b1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:19.441664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792a16a-21e0-4f91-84a0-9538cd9046b1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:19.443125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8792a16a-21e0-4f91-84a0-9538cd9046b1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-8792a16a-21e0-4f91-84a0-9538cd9046b1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 11:45:19.458284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:19.459488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:19.460635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:19.460635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:19.460635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 11:45:19.460635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 11:45:19.460635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 11:45:19.460635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 11:45:19.460635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 11:45:19.460635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 11:45:19.460635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 11:45:19.460635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:19.460635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 661/1280] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:19.465309 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1516a-f4ad-4fe8-bca9-bf26319e24d3 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:19.469619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:19.561502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1516a-f4ad-4fe8-bca9-bf26319e24d3 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:19.593826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d994e4-f319-4ccd-89f5-e446359c44ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:19.594420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d994e4-f319-4ccd-89f5-e446359c44ae None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:19.594703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d994e4-f319-4ccd-89f5-e446359c44ae None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:19.594981 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d994e4-f319-4ccd-89f5-e446359c44ae None admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:19.595803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d994e4-f319-4ccd-89f5-e446359c44ae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:19.596113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d994e4-f319-4ccd-89f5-e446359c44ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:19.597546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d994e4-f319-4ccd-89f5-e446359c44ae None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:19.614417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 620/1281] 10.222.0.67 () {56 vars in 1143 bytes} [Thu Jun 23 11:45:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:19.620126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792a16a-21e0-4f91-84a0-9538cd9046b1 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:19.707992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:19.717186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792a16a-21e0-4f91-84a0-9538cd9046b1 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:19.749616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45290d61-7050-4c7b-b5a1-bfd2251b0107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:19.750259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45290d61-7050-4c7b-b5a1-bfd2251b0107 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:19.750550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45290d61-7050-4c7b-b5a1-bfd2251b0107 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:19.750828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45290d61-7050-4c7b-b5a1-bfd2251b0107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:19.816401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45290d61-7050-4c7b-b5a1-bfd2251b0107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:19.818647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45290d61-7050-4c7b-b5a1-bfd2251b0107 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:19.908927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:19.919579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 662/1282] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 23 11:45:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 11:45:19.925982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d994e4-f319-4ccd-89f5-e446359c44ae None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:19.985960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:20.022227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d994e4-f319-4ccd-89f5-e446359c44ae None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:20.055475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91e798-f479-471f-9fc6-61525b5c2a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:20.056240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91e798-f479-471f-9fc6-61525b5c2a54 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:20.056524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91e798-f479-471f-9fc6-61525b5c2a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:20.056829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91e798-f479-471f-9fc6-61525b5c2a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:20.066667 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91e798-f479-471f-9fc6-61525b5c2a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:20.068086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91e798-f479-471f-9fc6-61525b5c2a54 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-eb91e798-f479-471f-9fc6-61525b5c2a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 11:45:20.084837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:20.085897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:20.087013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 11:45:20.087013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 11:45:20.087013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 11:45:20.087013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:20.087013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 621/1283] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:20.091776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45290d61-7050-4c7b-b5a1-bfd2251b0107 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:20.199964 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45290d61-7050-4c7b-b5a1-bfd2251b0107 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:20.233292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0af99c-2e36-40b1-8b3d-eed946e2f9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:20.234550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0af99c-2e36-40b1-8b3d-eed946e2f9ad None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:20.234835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0af99c-2e36-40b1-8b3d-eed946e2f9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:20.235116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0af99c-2e36-40b1-8b3d-eed946e2f9ad None admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:20.235838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0af99c-2e36-40b1-8b3d-eed946e2f9ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:20.236147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0af99c-2e36-40b1-8b3d-eed946e2f9ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:20.237542 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0af99c-2e36-40b1-8b3d-eed946e2f9ad None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:20.248782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 663/1284] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 23 11:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:20.254696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91e798-f479-471f-9fc6-61525b5c2a54 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:20.356455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91e798-f479-471f-9fc6-61525b5c2a54 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:20.372254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:20.394854 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648523f-fd84-4eb2-bab1-afb3ac41b963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:20.395643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648523f-fd84-4eb2-bab1-afb3ac41b963 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:20.395928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648523f-fd84-4eb2-bab1-afb3ac41b963 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:20.396205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a648523f-fd84-4eb2-bab1-afb3ac41b963 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:20.402796 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648523f-fd84-4eb2-bab1-afb3ac41b963 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:20.404236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a648523f-fd84-4eb2-bab1-afb3ac41b963 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application [None req-a648523f-fd84-4eb2-bab1-afb3ac41b963 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 11:45:20.418532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:20.419593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:20.420568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 11:45:20.420568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 11:45:20.420568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 11:45:20.420568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 11:45:20.420568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 11:45:20.420568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 11:45:20.420568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 11:45:20.420568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: ERROR keystone.server.flask.application Jun 23 11:45:20.420568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 622/1285] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 23 11:45:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:20.425766 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0af99c-2e36-40b1-8b3d-eed946e2f9ad None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:20.474860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:20.523265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0af99c-2e36-40b1-8b3d-eed946e2f9ad None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:20.555972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e96173-197c-41b0-9a09-cb1423683900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:20.556622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e96173-197c-41b0-9a09-cb1423683900 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:20.556943 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e96173-197c-41b0-9a09-cb1423683900 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:20.557226 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e96173-197c-41b0-9a09-cb1423683900 None admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:20.558012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e96173-197c-41b0-9a09-cb1423683900 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:20.558327 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e96173-197c-41b0-9a09-cb1423683900 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:20.559743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e96173-197c-41b0-9a09-cb1423683900 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:20.571588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 664/1286] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 23 11:45:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:45:20.578395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648523f-fd84-4eb2-bab1-afb3ac41b963 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:20.675844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a648523f-fd84-4eb2-bab1-afb3ac41b963 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:20.708051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d26e8-e760-4aea-b0a0-283f87475905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:20.708710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d26e8-e760-4aea-b0a0-283f87475905 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:20.708995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d26e8-e760-4aea-b0a0-283f87475905 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:20.709269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d26e8-e760-4aea-b0a0-283f87475905 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:20.710091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d26e8-e760-4aea-b0a0-283f87475905 None admin] RBAC: Adding query filter params (role.id=569707378794437b93d168fc8bb9cbc3, scope.project.id=19c2f661cc934aa7869e9b3d4efa875e, user.id=ee971347d23040d3984f961025e8157e) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:20.710394 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d26e8-e760-4aea-b0a0-283f87475905 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:20.711819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d26e8-e760-4aea-b0a0-283f87475905 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:20.717538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 623/1287] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 23 11:45:20 2022] GET /identity/v3/role_assignments?user.id=ee971347d23040d3984f961025e8157e&scope.project.id=19c2f661cc934aa7869e9b3d4efa875e&role.id=569707378794437b93d168fc8bb9cbc3 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:22.747148 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361ddb8-f70b-414d-848b-5bedfbe4dba0 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.747705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361ddb8-f70b-414d-848b-5bedfbe4dba0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.747705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361ddb8-f70b-414d-848b-5bedfbe4dba0 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.749111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 665/1288] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:45:22.754852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce688e-5854-4a6e-95d6-14643507921e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.755257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce688e-5854-4a6e-95d6-14643507921e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.755424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ce688e-5854-4a6e-95d6-14643507921e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.772520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e1ce688e-5854-4a6e-95d6-14643507921e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:22.773754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e1ce688e-5854-4a6e-95d6-14643507921e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:22.802290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:22.838218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 624/1289] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:45:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:45:22.860382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361ddb8-f70b-414d-848b-5bedfbe4dba0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:22.958434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361ddb8-f70b-414d-848b-5bedfbe4dba0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:22.969317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:22.993152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07554518-f785-4557-87ad-190ab7119f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:22.993941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07554518-f785-4557-87ad-190ab7119f39 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:22.994226 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07554518-f785-4557-87ad-190ab7119f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:22.994502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07554518-f785-4557-87ad-190ab7119f39 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:22.995151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07554518-f785-4557-87ad-190ab7119f39 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:22.996517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07554518-f785-4557-87ad-190ab7119f39 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-07554518-f785-4557-87ad-190ab7119f39 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:23.008274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:23.009530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:23.010591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 666/1290] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 23 11:45:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:23.015456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce688e-5854-4a6e-95d6-14643507921e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:23.110949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ce688e-5854-4a6e-95d6-14643507921e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:23.143191 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797768f9-10d5-4252-87b9-9bbf385f6608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:23.143844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797768f9-10d5-4252-87b9-9bbf385f6608 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:23.144130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797768f9-10d5-4252-87b9-9bbf385f6608 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:23.144408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797768f9-10d5-4252-87b9-9bbf385f6608 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:23.145149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797768f9-10d5-4252-87b9-9bbf385f6608 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:23.145455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797768f9-10d5-4252-87b9-9bbf385f6608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:23.146816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797768f9-10d5-4252-87b9-9bbf385f6608 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:23.151534 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 625/1291] 10.222.0.67 () {56 vars in 1116 bytes} [Thu Jun 23 11:45:23 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:23.157736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07554518-f785-4557-87ad-190ab7119f39 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:23.256390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07554518-f785-4557-87ad-190ab7119f39 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:23.289038 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d92b298-dd1f-4f10-9f09-0c6d8accaffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:23.289695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92b298-dd1f-4f10-9f09-0c6d8accaffb None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:23.289984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92b298-dd1f-4f10-9f09-0c6d8accaffb None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:23.290263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d92b298-dd1f-4f10-9f09-0c6d8accaffb None admin] PATH_INFO: `/v3/services` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:23.290972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92b298-dd1f-4f10-9f09-0c6d8accaffb None admin] RBAC: Adding query filter params (type=placement) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:23.291283 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92b298-dd1f-4f10-9f09-0c6d8accaffb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:23.292653 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d92b298-dd1f-4f10-9f09-0c6d8accaffb None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:23.297578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 667/1292] 10.222.0.67 () {56 vars in 1116 bytes} [Thu Jun 23 11:45:23 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:45:25.344861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87c84a-ad8c-4198-9ff6-da948bcb4e80 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:25.345480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87c84a-ad8c-4198-9ff6-da948bcb4e80 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:25.345480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87c84a-ad8c-4198-9ff6-da948bcb4e80 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:25.346799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 626/1293] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:45:25.351899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083146a-d4af-45d0-9727-cd667133ca2c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:25.352306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083146a-d4af-45d0-9727-cd667133ca2c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:25.352459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083146a-d4af-45d0-9727-cd667133ca2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:25.368959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b083146a-d4af-45d0-9727-cd667133ca2c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:25.370157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b083146a-d4af-45d0-9727-cd667133ca2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:25.403743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:25.447586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 668/1294] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:45:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 11:45:25.472570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87c84a-ad8c-4198-9ff6-da948bcb4e80 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:25.502748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:25.571738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87c84a-ad8c-4198-9ff6-da948bcb4e80 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:25.605859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70098338-fa11-4bdb-8f80-aa191e1f8334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:25.606446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70098338-fa11-4bdb-8f80-aa191e1f8334 None admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:25.606734 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70098338-fa11-4bdb-8f80-aa191e1f8334 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:25.607017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70098338-fa11-4bdb-8f80-aa191e1f8334 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:25.607746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70098338-fa11-4bdb-8f80-aa191e1f8334 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:25.609687 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70098338-fa11-4bdb-8f80-aa191e1f8334 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:25.622344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 627/1295] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 23 11:45:25 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:45:27.667622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443a65a-293d-43b4-b0f5-4a5e43327413 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:27.667999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443a65a-293d-43b4-b0f5-4a5e43327413 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:27.668197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443a65a-293d-43b4-b0f5-4a5e43327413 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:27.669707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 669/1296] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:45:27.674777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77381178-1c02-4863-96ed-907fa6e52e93 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:27.675188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77381178-1c02-4863-96ed-907fa6e52e93 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:27.675345 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77381178-1c02-4863-96ed-907fa6e52e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:27.692116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-77381178-1c02-4863-96ed-907fa6e52e93 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:27.693343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-77381178-1c02-4863-96ed-907fa6e52e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:27.763658 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 628/1297] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:45:27 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:45:27.787523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443a65a-293d-43b4-b0f5-4a5e43327413 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:27.886233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443a65a-293d-43b4-b0f5-4a5e43327413 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:27.918510 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85672cfe-4854-40a3-9284-a6b395f53f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:27.919237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85672cfe-4854-40a3-9284-a6b395f53f8b None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:27.919570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85672cfe-4854-40a3-9284-a6b395f53f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:27.919793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85672cfe-4854-40a3-9284-a6b395f53f8b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:27.920449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85672cfe-4854-40a3-9284-a6b395f53f8b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:27.922350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85672cfe-4854-40a3-9284-a6b395f53f8b None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-85672cfe-4854-40a3-9284-a6b395f53f8b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:27.933958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:27.935192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:27.936234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 670/1298] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 23 11:45:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:27.941688 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77381178-1c02-4863-96ed-907fa6e52e93 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:27.959475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:28.038733 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77381178-1c02-4863-96ed-907fa6e52e93 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.072453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c53067-462f-4a04-a86a-11cb50a73313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.073127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c53067-462f-4a04-a86a-11cb50a73313 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.073413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c53067-462f-4a04-a86a-11cb50a73313 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.073688 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c53067-462f-4a04-a86a-11cb50a73313 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.074387 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c53067-462f-4a04-a86a-11cb50a73313 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:28.074759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c53067-462f-4a04-a86a-11cb50a73313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.076043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c53067-462f-4a04-a86a-11cb50a73313 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:28.081066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 629/1299] 10.222.0.67 () {56 vars in 1116 bytes} [Thu Jun 23 11:45:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:28.086898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85672cfe-4854-40a3-9284-a6b395f53f8b None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.117803 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:28.185056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85672cfe-4854-40a3-9284-a6b395f53f8b None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.223701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94490d83-8f47-4c49-bea5-5a1a41035d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.224353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94490d83-8f47-4c49-bea5-5a1a41035d7a None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.224640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94490d83-8f47-4c49-bea5-5a1a41035d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.225038 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94490d83-8f47-4c49-bea5-5a1a41035d7a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.225728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94490d83-8f47-4c49-bea5-5a1a41035d7a None admin] RBAC: Adding query filter params (interface=public, service_id=9e5470113e184f4fb9fe280592cb8015, region_id=RegionOne) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:28.226037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94490d83-8f47-4c49-bea5-5a1a41035d7a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.227429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94490d83-8f47-4c49-bea5-5a1a41035d7a None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:28.232644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 671/1300] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 23 11:45:28 2022] GET /identity/v3/endpoints?service_id=9e5470113e184f4fb9fe280592cb8015&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:28.238868 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c53067-462f-4a04-a86a-11cb50a73313 None admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:28.321824 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:28.342907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c53067-462f-4a04-a86a-11cb50a73313 None admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:28.376758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd5c73-9425-4828-9119-1b19a66f39b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:28.377406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd5c73-9425-4828-9119-1b19a66f39b1 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:28.377765 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd5c73-9425-4828-9119-1b19a66f39b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:28.377964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd5c73-9425-4828-9119-1b19a66f39b1 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:28.378637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd5c73-9425-4828-9119-1b19a66f39b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:28.379989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd5c73-9425-4828-9119-1b19a66f39b1 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:28.388399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 630/1301] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 23 11:45:28 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:45:30.432634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f56af-6f7a-4743-aa33-11e960b92020 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:30.433242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f56af-6f7a-4743-aa33-11e960b92020 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:30.433242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f56af-6f7a-4743-aa33-11e960b92020 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:30.434595 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 672/1302] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:45:30.440132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211e587-7782-4743-87f7-90538c262003 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:30.440510 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211e587-7782-4743-87f7-90538c262003 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:30.440708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2211e587-7782-4743-87f7-90538c262003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:30.456931 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2211e587-7782-4743-87f7-90538c262003 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:45:30.458109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2211e587-7782-4743-87f7-90538c262003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:45:30.521384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 631/1303] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:45:30 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 11:45:30.545135 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f56af-6f7a-4743-aa33-11e960b92020 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:30.641142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:30.647975 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f56af-6f7a-4743-aa33-11e960b92020 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:30.682114 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5e16e-fca2-4655-869a-f87c1269a3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:30.682969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5e16e-fca2-4655-869a-f87c1269a3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:30.683202 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5e16e-fca2-4655-869a-f87c1269a3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:30.683478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd5e16e-fca2-4655-869a-f87c1269a3e6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:30.684127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5e16e-fca2-4655-869a-f87c1269a3e6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:30.685516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd5e16e-fca2-4655-869a-f87c1269a3e6 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application [None req-3cd5e16e-fca2-4655-869a-f87c1269a3e6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 11:45:30.697337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 11:45:30.698530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application with Lock( Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self._enter() Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application return self.creator() Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application created_value = creator( Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: ERROR keystone.server.flask.application Jun 23 11:45:30.699598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 673/1304] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 23 11:45:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 11:45:30.704496 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211e587-7782-4743-87f7-90538c262003 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:30.777718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:30.801327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2211e587-7782-4743-87f7-90538c262003 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:30.834563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d2715-f280-4c1e-b5ee-b09ba7f3d672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:30.835220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d2715-f280-4c1e-b5ee-b09ba7f3d672 None admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:30.835503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d2715-f280-4c1e-b5ee-b09ba7f3d672 None admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:30.835776 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d2715-f280-4c1e-b5ee-b09ba7f3d672 None admin] PATH_INFO: `/v3/services` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:30.836478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d2715-f280-4c1e-b5ee-b09ba7f3d672 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:45:30.836815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d2715-f280-4c1e-b5ee-b09ba7f3d672 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:30.838163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d2715-f280-4c1e-b5ee-b09ba7f3d672 None admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:30.843794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 632/1305] 10.222.0.67 () {56 vars in 1116 bytes} [Thu Jun 23 11:45:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:45:30.849524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5e16e-fca2-4655-869a-f87c1269a3e6 None admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:45:30.945909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd5e16e-fca2-4655-869a-f87c1269a3e6 None admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:45:30.978525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450ab9e-da70-447f-ac32-6918173fe2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:45:30.979163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450ab9e-da70-447f-ac32-6918173fe2a1 None admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:45:30.979446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450ab9e-da70-447f-ac32-6918173fe2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:45:30.979722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450ab9e-da70-447f-ac32-6918173fe2a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:45:30.980442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450ab9e-da70-447f-ac32-6918173fe2a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:45:30.981825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450ab9e-da70-447f-ac32-6918173fe2a1 None admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:45:30.998903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 674/1306] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 23 11:45:30 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:45:40.840188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=30 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:45:40.994793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:33.965708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512db7e-ac7f-45de-bf94-b00c45753c0b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:33.966263 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512db7e-ac7f-45de-bf94-b00c45753c0b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:33.966263 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512db7e-ac7f-45de-bf94-b00c45753c0b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:33.967612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 633/1307] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:46:33.973531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd61ed-67a5-4b85-805f-5423bf9de703 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:33.974016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd61ed-67a5-4b85-805f-5423bf9de703 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:33.974287 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd61ed-67a5-4b85-805f-5423bf9de703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:34.069904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4ebd61ed-67a5-4b85-805f-5423bf9de703 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:34.071071 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4ebd61ed-67a5-4b85-805f-5423bf9de703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:34.193602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 675/1308] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:46:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:34.244599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29729cfe-3940-4ac5-afee-8d12247f3f21 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:34.244949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29729cfe-3940-4ac5-afee-8d12247f3f21 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:34.245335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29729cfe-3940-4ac5-afee-8d12247f3f21 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:34.247148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 634/1309] 10.222.0.67 () {54 vars in 964 bytes} [Thu Jun 23 11:46:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:46:34.254708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cfaf0-24ed-4a3a-ba39-9a3396f3bb69 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:34.255460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cfaf0-24ed-4a3a-ba39-9a3396f3bb69 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:34.256314 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cfaf0-24ed-4a3a-ba39-9a3396f3bb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:34.372313 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3e0cfaf0-24ed-4a3a-ba39-9a3396f3bb69 None None] MFA Rules not processed for user `5f095d3370be457cb6687ac8f8c9f8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:34.373943 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3e0cfaf0-24ed-4a3a-ba39-9a3396f3bb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:34.403397 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:34.532890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29729cfe-3940-4ac5-afee-8d12247f3f21 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:34.535139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-29729cfe-3940-4ac5-afee-8d12247f3f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:34.540932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 676/1310] 10.222.0.67 () {58 vars in 1061 bytes} [Thu Jun 23 11:46:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:34.677125 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29729cfe-3940-4ac5-afee-8d12247f3f21 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:34.757073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fca9bb-1a63-4c08-a433-46f5f6fdaa6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:34.757993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fca9bb-1a63-4c08-a433-46f5f6fdaa6f service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:34.758284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fca9bb-1a63-4c08-a433-46f5f6fdaa6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:34.758567 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fca9bb-1a63-4c08-a433-46f5f6fdaa6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:34.760242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-21fca9bb-1a63-4c08-a433-46f5f6fdaa6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:34.820791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:34.880925 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fca9bb-1a63-4c08-a433-46f5f6fdaa6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:34.882371 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fca9bb-1a63-4c08-a433-46f5f6fdaa6f service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:34.903666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 635/1311] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:37.274777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8d1ab-ba49-422b-8882-9fa0853c0b24 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:37.275307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8d1ab-ba49-422b-8882-9fa0853c0b24 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:37.275307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8d1ab-ba49-422b-8882-9fa0853c0b24 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:37.277072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 677/1312] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:46:37.282695 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6a783-e272-4c0b-b98a-cac29f6c655b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:37.283115 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6a783-e272-4c0b-b98a-cac29f6c655b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:37.283289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6a783-e272-4c0b-b98a-cac29f6c655b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:37.360034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-96e6a783-e272-4c0b-b98a-cac29f6c655b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:37.362140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-96e6a783-e272-4c0b-b98a-cac29f6c655b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:37.483041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 636/1313] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:46:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:37.537059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff714cfd-32b0-4438-8be3-f4272e427daa None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:37.537356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff714cfd-32b0-4438-8be3-f4272e427daa None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:37.537639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff714cfd-32b0-4438-8be3-f4272e427daa None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:37.539143 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 678/1314] 10.222.0.67 () {54 vars in 964 bytes} [Thu Jun 23 11:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:46:37.545132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35c8e0-6cb6-4d13-a15b-d7942a8c344c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:37.545507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35c8e0-6cb6-4d13-a15b-d7942a8c344c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:37.545871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35c8e0-6cb6-4d13-a15b-d7942a8c344c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:37.557274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:37.624167 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ee35c8e0-6cb6-4d13-a15b-d7942a8c344c None None] MFA Rules not processed for user `5f095d3370be457cb6687ac8f8c9f8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:37.625435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ee35c8e0-6cb6-4d13-a15b-d7942a8c344c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:37.717488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 637/1315] 10.222.0.67 () {58 vars in 1061 bytes} [Thu Jun 23 11:46:37 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:37.727218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff714cfd-32b0-4438-8be3-f4272e427daa None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:37.729284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ff714cfd-32b0-4438-8be3-f4272e427daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:37.844087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff714cfd-32b0-4438-8be3-f4272e427daa None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:37.915333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:37.918062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0071612a-d2f6-484b-8cab-740946ac2265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:37.918936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071612a-d2f6-484b-8cab-740946ac2265 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:37.919407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071612a-d2f6-484b-8cab-740946ac2265 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:37.919879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0071612a-d2f6-484b-8cab-740946ac2265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:37.921759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0071612a-d2f6-484b-8cab-740946ac2265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:38.027763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071612a-d2f6-484b-8cab-740946ac2265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:38.029127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0071612a-d2f6-484b-8cab-740946ac2265 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:38.047183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 679/1316] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:40.581264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78617c-b96b-4f66-a69c-55a8b1350d2d None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:40.581806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78617c-b96b-4f66-a69c-55a8b1350d2d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:40.581806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78617c-b96b-4f66-a69c-55a8b1350d2d None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:40.583402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 638/1317] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:46:40.592966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1de528-74fa-4887-b7e2-4e363e2e1b90 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:40.594791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1de528-74fa-4887-b7e2-4e363e2e1b90 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:40.595083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1de528-74fa-4887-b7e2-4e363e2e1b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:40.659374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4f1de528-74fa-4887-b7e2-4e363e2e1b90 None None] MFA Rules not processed for user `6dcf44525c2549deba53d9ec8dbe03ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:40.660589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4f1de528-74fa-4887-b7e2-4e363e2e1b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:40.712532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:40.757240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 680/1318] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:46:40 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:46:40.796214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78617c-b96b-4f66-a69c-55a8b1350d2d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:40.798296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bc78617c-b96b-4f66-a69c-55a8b1350d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:40.901438 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78617c-b96b-4f66-a69c-55a8b1350d2d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:40.904146 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:40.981084 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363712c-157d-4060-9c39-e610ed815998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:40.981734 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363712c-157d-4060-9c39-e610ed815998 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:40.982018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363712c-157d-4060-9c39-e610ed815998 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:40.982295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363712c-157d-4060-9c39-e610ed815998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:40.984660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c363712c-157d-4060-9c39-e610ed815998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:41.098769 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363712c-157d-4060-9c39-e610ed815998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:41.100210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363712c-157d-4060-9c39-e610ed815998 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:41.118472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 639/1319] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:43.730068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1795ca-a60e-45f0-a75f-42cd53e41cda None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:43.730572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1795ca-a60e-45f0-a75f-42cd53e41cda None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:43.730663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1795ca-a60e-45f0-a75f-42cd53e41cda None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:43.732027 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 681/1320] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:46:43.737669 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41013-4c56-40d0-b854-89bd1c93913d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:43.738048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41013-4c56-40d0-b854-89bd1c93913d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:43.738235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf41013-4c56-40d0-b854-89bd1c93913d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:43.777255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:43.803809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7cf41013-4c56-40d0-b854-89bd1c93913d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:43.805052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7cf41013-4c56-40d0-b854-89bd1c93913d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:43.898799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 640/1321] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:46:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:43.938699 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1795ca-a60e-45f0-a75f-42cd53e41cda None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:43.940276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7d1795ca-a60e-45f0-a75f-42cd53e41cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:44.047301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1795ca-a60e-45f0-a75f-42cd53e41cda None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:44.119341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5362581-15eb-4204-8ae4-30a38cc5294b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:44.120011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5362581-15eb-4204-8ae4-30a38cc5294b service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:44.120299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5362581-15eb-4204-8ae4-30a38cc5294b service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:44.120579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5362581-15eb-4204-8ae4-30a38cc5294b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:44.122263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d5362581-15eb-4204-8ae4-30a38cc5294b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:44.129053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:44.231733 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5362581-15eb-4204-8ae4-30a38cc5294b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:44.233193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5362581-15eb-4204-8ae4-30a38cc5294b service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:44.252080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 682/1322] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:46.974459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f31b8-b88a-4f18-844a-dc2917e77ce5 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:46.974980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f31b8-b88a-4f18-844a-dc2917e77ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:46.975075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f31b8-b88a-4f18-844a-dc2917e77ce5 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:46.976404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 641/1323] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:46:46.981906 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146f137-aad1-42af-bc88-9c664d9d2e4a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:46.982338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146f137-aad1-42af-bc88-9c664d9d2e4a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:46.982480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2146f137-aad1-42af-bc88-9c664d9d2e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:47.049077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2146f137-aad1-42af-bc88-9c664d9d2e4a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:47.050347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2146f137-aad1-42af-bc88-9c664d9d2e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:47.111803 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:47.157897 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 683/1324] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:47.194972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f31b8-b88a-4f18-844a-dc2917e77ce5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:47.196665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4f8f31b8-b88a-4f18-844a-dc2917e77ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:47.316898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f31b8-b88a-4f18-844a-dc2917e77ce5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:47.330468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:47.397052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f679c6-8719-436c-bf0a-e1c134c983bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:47.397777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f679c6-8719-436c-bf0a-e1c134c983bb service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:47.398097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f679c6-8719-436c-bf0a-e1c134c983bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:47.398501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f679c6-8719-436c-bf0a-e1c134c983bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:47.400481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b6f679c6-8719-436c-bf0a-e1c134c983bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:47.517396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f679c6-8719-436c-bf0a-e1c134c983bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:47.518734 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f679c6-8719-436c-bf0a-e1c134c983bb service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:47.536889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 642/1325] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:49.645105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196be97-04c0-4df6-b727-f490b728d393 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:49.645660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196be97-04c0-4df6-b727-f490b728d393 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:49.645660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196be97-04c0-4df6-b727-f490b728d393 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:49.646969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 684/1326] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:46:49.652684 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb5af2-9d95-4a74-a59d-f7de8fcfd26c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:49.653208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb5af2-9d95-4a74-a59d-f7de8fcfd26c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:49.653300 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb5af2-9d95-4a74-a59d-f7de8fcfd26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:49.704867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:49.725665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-79eb5af2-9d95-4a74-a59d-f7de8fcfd26c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:49.726903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-79eb5af2-9d95-4a74-a59d-f7de8fcfd26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:49.827113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 643/1327] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:49.863551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196be97-04c0-4df6-b727-f490b728d393 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:49.865149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5196be97-04c0-4df6-b727-f490b728d393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:49.969021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196be97-04c0-4df6-b727-f490b728d393 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:50.045593 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b59d96-cbcc-432f-b9b1-611effd3892d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:50.046278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b59d96-cbcc-432f-b9b1-611effd3892d service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:50.046567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b59d96-cbcc-432f-b9b1-611effd3892d service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:50.046874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b59d96-cbcc-432f-b9b1-611effd3892d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:50.048700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a5b59d96-cbcc-432f-b9b1-611effd3892d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:50.066780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:50.161681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b59d96-cbcc-432f-b9b1-611effd3892d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:50.163111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b59d96-cbcc-432f-b9b1-611effd3892d service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:50.181229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 685/1328] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:52.277084 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402fb35-a522-4d74-a081-fe105b24b35a None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:52.277651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402fb35-a522-4d74-a081-fe105b24b35a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:52.277651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402fb35-a522-4d74-a081-fe105b24b35a None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:52.279034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 644/1329] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:46:52.284942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008052e-27b3-4fac-97ee-ff346f17b5cf None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:52.285365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008052e-27b3-4fac-97ee-ff346f17b5cf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:52.285511 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008052e-27b3-4fac-97ee-ff346f17b5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:52.351309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7008052e-27b3-4fac-97ee-ff346f17b5cf None None] MFA Rules not processed for user `6dcf44525c2549deba53d9ec8dbe03ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:52.352467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7008052e-27b3-4fac-97ee-ff346f17b5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:52.420962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:52.455278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 686/1330] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:46:52 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:52.493732 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402fb35-a522-4d74-a081-fe105b24b35a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:52.495267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3402fb35-a522-4d74-a081-fe105b24b35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:52.600539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402fb35-a522-4d74-a081-fe105b24b35a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:52.623438 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:52.685041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577679b6-1bf1-46e6-ba88-3e81b4f8a08c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:52.685713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577679b6-1bf1-46e6-ba88-3e81b4f8a08c service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:52.685999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577679b6-1bf1-46e6-ba88-3e81b4f8a08c service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:52.686274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577679b6-1bf1-46e6-ba88-3e81b4f8a08c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:52.687940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-577679b6-1bf1-46e6-ba88-3e81b4f8a08c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:52.798597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577679b6-1bf1-46e6-ba88-3e81b4f8a08c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:52.800038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577679b6-1bf1-46e6-ba88-3e81b4f8a08c service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:52.818760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 645/1331] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:52 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:55.115352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7840af-52f8-46f5-bd2c-a4c99137c847 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:55.115924 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7840af-52f8-46f5-bd2c-a4c99137c847 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:55.115924 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7840af-52f8-46f5-bd2c-a4c99137c847 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:55.117357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 687/1332] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:46:55.125691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7cf8c-22a1-461d-b89e-38cca88f2361 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:55.126168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7cf8c-22a1-461d-b89e-38cca88f2361 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:55.126263 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf7cf8c-22a1-461d-b89e-38cca88f2361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:55.190180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:55.193385 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7cf7cf8c-22a1-461d-b89e-38cca88f2361 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:55.195175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7cf7cf8c-22a1-461d-b89e-38cca88f2361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:55.290288 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 646/1333] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:55.326162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7840af-52f8-46f5-bd2c-a4c99137c847 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:55.327688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5d7840af-52f8-46f5-bd2c-a4c99137c847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:55.430696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7840af-52f8-46f5-bd2c-a4c99137c847 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:55.502748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc730c7-7b38-42e4-a25f-21ad90b3c744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:55.503401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc730c7-7b38-42e4-a25f-21ad90b3c744 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:55.503688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc730c7-7b38-42e4-a25f-21ad90b3c744 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:55.503969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc730c7-7b38-42e4-a25f-21ad90b3c744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:55.505682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fbc730c7-7b38-42e4-a25f-21ad90b3c744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:55.534425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:55.617081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc730c7-7b38-42e4-a25f-21ad90b3c744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:55.618521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc730c7-7b38-42e4-a25f-21ad90b3c744 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:55.638332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 688/1334] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:46:57.714789 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb1b1f-ee6f-45a4-8a57-d4487cb01020 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:57.715364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb1b1f-ee6f-45a4-8a57-d4487cb01020 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:57.715364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb1b1f-ee6f-45a4-8a57-d4487cb01020 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:57.716802 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 647/1335] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:46:57.722742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16192985-a719-4c0d-9fce-681cad83c49c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:57.723212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16192985-a719-4c0d-9fce-681cad83c49c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:57.723327 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16192985-a719-4c0d-9fce-681cad83c49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:57.787949 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-16192985-a719-4c0d-9fce-681cad83c49c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:46:57.789165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-16192985-a719-4c0d-9fce-681cad83c49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:57.874753 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:57.899218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 689/1336] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:46:57.939981 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb1b1f-ee6f-45a4-8a57-d4487cb01020 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:46:57.941567 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d6bb1b1f-ee6f-45a4-8a57-d4487cb01020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:58.047040 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb1b1f-ee6f-45a4-8a57-d4487cb01020 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:46:58.075111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:46:58.119807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6a20f-cd93-4ae0-a03e-b0c7b1ed7008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:46:58.120450 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6a20f-cd93-4ae0-a03e-b0c7b1ed7008 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:46:58.120826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6a20f-cd93-4ae0-a03e-b0c7b1ed7008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:46:58.121048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6a20f-cd93-4ae0-a03e-b0c7b1ed7008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:46:58.122709 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d6c6a20f-cd93-4ae0-a03e-b0c7b1ed7008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:46:58.229501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6a20f-cd93-4ae0-a03e-b0c7b1ed7008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:46:58.230951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6a20f-cd93-4ae0-a03e-b0c7b1ed7008 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:46:58.249711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 648/1337] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:46:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:00.961107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a809ba-1aef-4315-96c4-cf05c8f16d79 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:00.961619 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a809ba-1aef-4315-96c4-cf05c8f16d79 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:00.961619 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a809ba-1aef-4315-96c4-cf05c8f16d79 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:00.962963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 690/1338] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:00.968369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac47d45-a357-49e9-be0c-41419fcae862 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:00.968857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac47d45-a357-49e9-be0c-41419fcae862 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:00.968968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac47d45-a357-49e9-be0c-41419fcae862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:01.039066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-cac47d45-a357-49e9-be0c-41419fcae862 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:01.040320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cac47d45-a357-49e9-be0c-41419fcae862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:01.049436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:01.142998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 649/1339] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:01.181059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a809ba-1aef-4315-96c4-cf05c8f16d79 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:01.182632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c9a809ba-1aef-4315-96c4-cf05c8f16d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:01.298257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a809ba-1aef-4315-96c4-cf05c8f16d79 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:01.379427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a5475-a438-4273-8e94-b4676f9243fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:01.380087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a5475-a438-4273-8e94-b4676f9243fc service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:01.380372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a5475-a438-4273-8e94-b4676f9243fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:01.380654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a5475-a438-4273-8e94-b4676f9243fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:01.382381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-630a5475-a438-4273-8e94-b4676f9243fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:01.422747 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:01.502524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a5475-a438-4273-8e94-b4676f9243fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:01.503969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a5475-a438-4273-8e94-b4676f9243fc service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:01.523099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 691/1340] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:05.757316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bad0d9-9834-4555-9007-cb4c52d7b48c None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:05.757316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bad0d9-9834-4555-9007-cb4c52d7b48c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:05.758159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bad0d9-9834-4555-9007-cb4c52d7b48c None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:05.758822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 650/1341] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:05.764722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51193cf5-2b03-4f43-8816-acaf7d26ec84 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:05.765284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51193cf5-2b03-4f43-8816-acaf7d26ec84 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:05.765284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51193cf5-2b03-4f43-8816-acaf7d26ec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:05.881531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-51193cf5-2b03-4f43-8816-acaf7d26ec84 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:05.883080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-51193cf5-2b03-4f43-8816-acaf7d26ec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:06.024955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:06.043277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 692/1342] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:06.083468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bad0d9-9834-4555-9007-cb4c52d7b48c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:06.086115 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-52bad0d9-9834-4555-9007-cb4c52d7b48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:06.207442 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bad0d9-9834-4555-9007-cb4c52d7b48c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:06.211179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dee85b-cae0-416c-97b0-f192c502844b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:06.211843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dee85b-cae0-416c-97b0-f192c502844b service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:06.212131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dee85b-cae0-416c-97b0-f192c502844b service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:06.212418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dee85b-cae0-416c-97b0-f192c502844b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:06.223160 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dee85b-cae0-416c-97b0-f192c502844b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:06.224632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dee85b-cae0-416c-97b0-f192c502844b service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:06.238304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 651/1343] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:09.433469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c89be-c037-4831-bb82-a65ac4e8c5cc None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:09.434538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c89be-c037-4831-bb82-a65ac4e8c5cc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:09.434538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301c89be-c037-4831-bb82-a65ac4e8c5cc None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:09.437290 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 693/1344] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:47:09.444793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e98fb-7c89-4ab8-b0c5-2b941696c114 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:09.446878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e98fb-7c89-4ab8-b0c5-2b941696c114 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:09.446878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89e98fb-7c89-4ab8-b0c5-2b941696c114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:09.481521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d89e98fb-7c89-4ab8-b0c5-2b941696c114 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:09.482790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d89e98fb-7c89-4ab8-b0c5-2b941696c114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:09.514465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:09.571197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 652/1345] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:09.611518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c89be-c037-4831-bb82-a65ac4e8c5cc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:09.622618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301c89be-c037-4831-bb82-a65ac4e8c5cc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:09.626348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f208e-c078-4d8b-b05b-5d3b142519e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:09.627089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f208e-c078-4d8b-b05b-5d3b142519e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:09.627448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f208e-c078-4d8b-b05b-5d3b142519e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:09.627802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f208e-c078-4d8b-b05b-5d3b142519e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:09.637898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f208e-c078-4d8b-b05b-5d3b142519e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:09.639353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f208e-c078-4d8b-b05b-5d3b142519e5 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:09.650007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 694/1346] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:14.515543 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d0245-b898-4ca1-8c94-77401a4ddc4b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:14.516108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d0245-b898-4ca1-8c94-77401a4ddc4b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:14.516108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d0245-b898-4ca1-8c94-77401a4ddc4b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:14.517461 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 653/1347] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:14.522742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aec725-da1f-40ae-9f7f-7e136309146e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:14.523154 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aec725-da1f-40ae-9f7f-7e136309146e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:14.523421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aec725-da1f-40ae-9f7f-7e136309146e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:14.541568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e8aec725-da1f-40ae-9f7f-7e136309146e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:14.542786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e8aec725-da1f-40ae-9f7f-7e136309146e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:14.550837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 695/1348] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:14.593393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d0245-b898-4ca1-8c94-77401a4ddc4b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:14.604207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d0245-b898-4ca1-8c94-77401a4ddc4b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:14.607785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e0221-c379-44b4-8d8c-477be753f632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:14.608489 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e0221-c379-44b4-8d8c-477be753f632 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:14.608816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e0221-c379-44b4-8d8c-477be753f632 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:14.609116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e0221-c379-44b4-8d8c-477be753f632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:14.618506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e0221-c379-44b4-8d8c-477be753f632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:14.619978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e0221-c379-44b4-8d8c-477be753f632 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:14.629850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 654/1349] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:19.659212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21173f-94f6-48ef-a0fa-a678c9086e10 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:19.660093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21173f-94f6-48ef-a0fa-a678c9086e10 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:19.660510 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21173f-94f6-48ef-a0fa-a678c9086e10 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:19.662889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 696/1350] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 11:47:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:19.669048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a4bf6-c310-4bee-a2ae-5d99d64617ca None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:19.669564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a4bf6-c310-4bee-a2ae-5d99d64617ca None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:19.669727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a4bf6-c310-4bee-a2ae-5d99d64617ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:19.775525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9a1a4bf6-c310-4bee-a2ae-5d99d64617ca None None] MFA Rules not processed for user `ee971347d23040d3984f961025e8157e`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:19.777426 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9a1a4bf6-c310-4bee-a2ae-5d99d64617ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:19.915099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 655/1351] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 11:47:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:24.434204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2c6fe-a45a-4a11-8cd8-25d2464888ea None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:24.434204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2c6fe-a45a-4a11-8cd8-25d2464888ea None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:24.434204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde2c6fe-a45a-4a11-8cd8-25d2464888ea None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:24.435654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 697/1352] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 11:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:24.442710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13531ee9-8f2a-4822-a829-a6c24f2d1aed None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:24.443241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13531ee9-8f2a-4822-a829-a6c24f2d1aed None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:24.443830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13531ee9-8f2a-4822-a829-a6c24f2d1aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:24.465934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-13531ee9-8f2a-4822-a829-a6c24f2d1aed None None] MFA Rules not processed for user `ee971347d23040d3984f961025e8157e`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:24.467439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-13531ee9-8f2a-4822-a829-a6c24f2d1aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:24.482042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 656/1353] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 11:47:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:24.536220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=13 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:26.405173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30167f77-30ee-4445-b658-cbf12783e05e None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:26.405834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30167f77-30ee-4445-b658-cbf12783e05e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:26.406026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30167f77-30ee-4445-b658-cbf12783e05e None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:26.407666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 698/1354] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 11:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:26.414748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0c9c7-dadb-4a9c-9196-7f2f56a8b596 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:26.415368 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0c9c7-dadb-4a9c-9196-7f2f56a8b596 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:26.415540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0c9c7-dadb-4a9c-9196-7f2f56a8b596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:26.434867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2da0c9c7-dadb-4a9c-9196-7f2f56a8b596 None None] MFA Rules not processed for user `ee971347d23040d3984f961025e8157e`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:26.436108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2da0c9c7-dadb-4a9c-9196-7f2f56a8b596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:26.445990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 657/1355] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 11:47:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:30.127826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce62ba-e914-4b08-b3b8-b62ba374df4f None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:30.128770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce62ba-e914-4b08-b3b8-b62ba374df4f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:30.129004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fce62ba-e914-4b08-b3b8-b62ba374df4f None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:30.135489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 699/1356] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:30.136954 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fcfb7-b5af-49cc-a807-472bd7d8b3c1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:30.137729 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fcfb7-b5af-49cc-a807-472bd7d8b3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:30.138189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93fcfb7-b5af-49cc-a807-472bd7d8b3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:30.178463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b93fcfb7-b5af-49cc-a807-472bd7d8b3c1 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:30.181373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b93fcfb7-b5af-49cc-a807-472bd7d8b3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:30.194129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 658/1357] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:30.262083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf941c-1275-4361-a092-d06cf9375d79 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:30.262376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf941c-1275-4361-a092-d06cf9375d79 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:30.262662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf941c-1275-4361-a092-d06cf9375d79 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:30.264058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 700/1358] 10.222.0.67 () {54 vars in 961 bytes} [Thu Jun 23 11:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:30.275468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb78c7-4cb0-4605-bb35-b6f8c428cc67 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:30.277405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb78c7-4cb0-4605-bb35-b6f8c428cc67 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:30.278169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb78c7-4cb0-4605-bb35-b6f8c428cc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:30.338520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:30.348348 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-badb78c7-4cb0-4605-bb35-b6f8c428cc67 None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:30.349749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-badb78c7-4cb0-4605-bb35-b6f8c428cc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:30.443032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 659/1359] 10.222.0.67 () {58 vars in 1058 bytes} [Thu Jun 23 11:47:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:30.451120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf941c-1275-4361-a092-d06cf9375d79 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:30.466875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf941c-1275-4361-a092-d06cf9375d79 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:30.472467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913b6e4-8824-4685-a964-22107ec8b836 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:30.473165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913b6e4-8824-4685-a964-22107ec8b836 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:30.473453 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913b6e4-8824-4685-a964-22107ec8b836 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:30.473730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913b6e4-8824-4685-a964-22107ec8b836 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:30.482458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913b6e4-8824-4685-a964-22107ec8b836 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:30.484020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913b6e4-8824-4685-a964-22107ec8b836 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:30.494210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 701/1360] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:32.765702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3df733-194c-4c9a-ac7b-3cdd0435c122 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:32.766321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3df733-194c-4c9a-ac7b-3cdd0435c122 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:32.766321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3df733-194c-4c9a-ac7b-3cdd0435c122 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:32.767641 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 660/1361] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:32.773526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11300eec-f16d-4fa6-8985-f841dda3d791 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:32.774010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11300eec-f16d-4fa6-8985-f841dda3d791 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:32.774092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11300eec-f16d-4fa6-8985-f841dda3d791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:32.793687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-11300eec-f16d-4fa6-8985-f841dda3d791 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:32.795275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-11300eec-f16d-4fa6-8985-f841dda3d791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:32.804349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 702/1362] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:32.850070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3df733-194c-4c9a-ac7b-3cdd0435c122 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:32.866286 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3df733-194c-4c9a-ac7b-3cdd0435c122 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:32.871937 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48e7b3-081f-471e-ab10-3f8fc71ccb33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:32.872598 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48e7b3-081f-471e-ab10-3f8fc71ccb33 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:32.872909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48e7b3-081f-471e-ab10-3f8fc71ccb33 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:32.873195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48e7b3-081f-471e-ab10-3f8fc71ccb33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:32.882470 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48e7b3-081f-471e-ab10-3f8fc71ccb33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:32.884149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48e7b3-081f-471e-ab10-3f8fc71ccb33 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:32.894852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 661/1363] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:35.296796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a197f-dffe-419a-b55e-053065967562 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:35.297349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a197f-dffe-419a-b55e-053065967562 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:35.297457 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a197f-dffe-419a-b55e-053065967562 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:35.298817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 703/1364] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:35.304360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf2705-db59-40e9-ae16-7c36e679280f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:35.305178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf2705-db59-40e9-ae16-7c36e679280f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:35.305464 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf2705-db59-40e9-ae16-7c36e679280f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:35.329371 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-24cf2705-db59-40e9-ae16-7c36e679280f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:35.330719 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-24cf2705-db59-40e9-ae16-7c36e679280f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:35.339657 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 662/1365] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:47:35.390864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b5060-87eb-4d42-8523-f8b8c02981ef None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:35.391157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b5060-87eb-4d42-8523-f8b8c02981ef None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:35.391449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b5060-87eb-4d42-8523-f8b8c02981ef None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:35.392884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 704/1366] 10.222.0.67 () {54 vars in 961 bytes} [Thu Jun 23 11:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:35.398911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdd337-ee02-4320-b68b-71fc37a49398 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:35.399202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdd337-ee02-4320-b68b-71fc37a49398 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:35.399485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdd337-ee02-4320-b68b-71fc37a49398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:35.422234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1bfdd337-ee02-4320-b68b-71fc37a49398 None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:35.423454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1bfdd337-ee02-4320-b68b-71fc37a49398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:35.433276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 663/1367] 10.222.0.67 () {58 vars in 1058 bytes} [Thu Jun 23 11:47:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:35.441284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b5060-87eb-4d42-8523-f8b8c02981ef None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:35.455636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b5060-87eb-4d42-8523-f8b8c02981ef None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:35.461095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d2f50-1e3e-4da4-bcfb-baff772c6950 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:35.461804 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2f50-1e3e-4da4-bcfb-baff772c6950 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:35.462145 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2f50-1e3e-4da4-bcfb-baff772c6950 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:35.462491 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d2f50-1e3e-4da4-bcfb-baff772c6950 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:35.471666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d2f50-1e3e-4da4-bcfb-baff772c6950 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:35.474066 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d2f50-1e3e-4da4-bcfb-baff772c6950 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:35.484449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 705/1368] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:37.615830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6890-639f-4336-b35b-ab98c17997e9 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:37.616371 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6890-639f-4336-b35b-ab98c17997e9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:37.616844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60a6890-639f-4336-b35b-ab98c17997e9 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:37.619370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 664/1369] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:37.625873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bca60-fe91-41af-81c6-55a11e029799 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:37.626355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bca60-fe91-41af-81c6-55a11e029799 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:37.626437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616bca60-fe91-41af-81c6-55a11e029799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:37.645201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-616bca60-fe91-41af-81c6-55a11e029799 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:37.646425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-616bca60-fe91-41af-81c6-55a11e029799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:37.667814 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 706/1370] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:37.704116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a6890-639f-4336-b35b-ab98c17997e9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:37.721172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60a6890-639f-4336-b35b-ab98c17997e9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:37.727643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820a865-910c-4261-9c83-f52f73ba852f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:37.728472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820a865-910c-4261-9c83-f52f73ba852f service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:37.728902 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820a865-910c-4261-9c83-f52f73ba852f service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:37.729289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820a865-910c-4261-9c83-f52f73ba852f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:37.739870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820a865-910c-4261-9c83-f52f73ba852f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:37.741976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820a865-910c-4261-9c83-f52f73ba852f service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:37.753319 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 665/1371] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:38.494148 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28613c0-1f28-44b8-a478-d88b8e2ed88a None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:38.494436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28613c0-1f28-44b8-a478-d88b8e2ed88a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:38.494934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28613c0-1f28-44b8-a478-d88b8e2ed88a None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:38.496317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 707/1372] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:38.501681 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109eb717-ca37-48db-842a-a75e86c09e0b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:38.501972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109eb717-ca37-48db-842a-a75e86c09e0b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:38.502422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109eb717-ca37-48db-842a-a75e86c09e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:38.533703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-109eb717-ca37-48db-842a-a75e86c09e0b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:38.534967 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-109eb717-ca37-48db-842a-a75e86c09e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:38.543654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 666/1373] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:38.891991 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c02070-2d24-4a9a-aa28-d89717c74a95 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:38.891991 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c02070-2d24-4a9a-aa28-d89717c74a95 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:38.891991 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c02070-2d24-4a9a-aa28-d89717c74a95 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:38.893576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 708/1374] 10.222.0.67 () {54 vars in 963 bytes} [Thu Jun 23 11:47:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:38.898452 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed355a-ddfc-47de-bc2c-2f917851463b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:38.899082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed355a-ddfc-47de-bc2c-2f917851463b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:38.899082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed355a-ddfc-47de-bc2c-2f917851463b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:38.952515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-09ed355a-ddfc-47de-bc2c-2f917851463b None None] MFA Rules not processed for user `abcbb404d9a148568a82697abd51ec30`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:38.953754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-09ed355a-ddfc-47de-bc2c-2f917851463b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:39.018427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 667/1375] 10.222.0.67 () {58 vars in 1060 bytes} [Thu Jun 23 11:47:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:39.026122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c02070-2d24-4a9a-aa28-d89717c74a95 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:39.037564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c02070-2d24-4a9a-aa28-d89717c74a95 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:39.041098 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007b250-2d06-4113-b9ed-a5e9c5bfbdbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:39.041702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007b250-2d06-4113-b9ed-a5e9c5bfbdbf service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:39.041992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007b250-2d06-4113-b9ed-a5e9c5bfbdbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:39.042273 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007b250-2d06-4113-b9ed-a5e9c5bfbdbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:39.052606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007b250-2d06-4113-b9ed-a5e9c5bfbdbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:39.054144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007b250-2d06-4113-b9ed-a5e9c5bfbdbf service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:39.066523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 709/1376] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:39.854243 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e1f9a-58fa-4e28-9ff7-217064098209 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:39.854544 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e1f9a-58fa-4e28-9ff7-217064098209 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:39.854948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6e1f9a-58fa-4e28-9ff7-217064098209 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:39.856363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 668/1377] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:39.861883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c5f4f-ab93-4674-b233-45f3aea0ed11 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:39.863041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c5f4f-ab93-4674-b233-45f3aea0ed11 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:39.863041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1c5f4f-ab93-4674-b233-45f3aea0ed11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:39.883080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0d1c5f4f-ab93-4674-b233-45f3aea0ed11 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:39.884516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0d1c5f4f-ab93-4674-b233-45f3aea0ed11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:39.893830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 710/1378] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:39.939658 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e1f9a-58fa-4e28-9ff7-217064098209 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:39.954714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6e1f9a-58fa-4e28-9ff7-217064098209 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:39.960275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf79e9-df9f-42f0-adc8-6a02fab562d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:39.961060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf79e9-df9f-42f0-adc8-6a02fab562d7 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:39.961412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf79e9-df9f-42f0-adc8-6a02fab562d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:39.961758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf79e9-df9f-42f0-adc8-6a02fab562d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:39.970843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf79e9-df9f-42f0-adc8-6a02fab562d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:39.972491 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf79e9-df9f-42f0-adc8-6a02fab562d7 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:39.976413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:39.983051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 669/1379] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:42.039103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39594b99-736c-4ab8-aa4c-0b530bd29323 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:42.039711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39594b99-736c-4ab8-aa4c-0b530bd29323 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:42.039711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39594b99-736c-4ab8-aa4c-0b530bd29323 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:42.041151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 711/1380] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:42.047094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af25fe-88bb-4d86-a431-95864665dda3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:42.047561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af25fe-88bb-4d86-a431-95864665dda3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:42.047702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af25fe-88bb-4d86-a431-95864665dda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:42.066872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e2af25fe-88bb-4d86-a431-95864665dda3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:42.068094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e2af25fe-88bb-4d86-a431-95864665dda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:42.081022 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 670/1381] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:42.130144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39594b99-736c-4ab8-aa4c-0b530bd29323 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:42.145823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39594b99-736c-4ab8-aa4c-0b530bd29323 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:42.151206 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60667880-8e3b-41b0-b59d-dadf765ed60a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:42.151845 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60667880-8e3b-41b0-b59d-dadf765ed60a service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:42.152132 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60667880-8e3b-41b0-b59d-dadf765ed60a service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:42.152409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60667880-8e3b-41b0-b59d-dadf765ed60a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:42.160744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60667880-8e3b-41b0-b59d-dadf765ed60a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:42.162587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60667880-8e3b-41b0-b59d-dadf765ed60a service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:42.178720 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 712/1382] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:44.272057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539bdbd-4bfd-49fd-b415-c97c954be3ef None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:44.272651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539bdbd-4bfd-49fd-b415-c97c954be3ef None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:44.272974 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539bdbd-4bfd-49fd-b415-c97c954be3ef None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:44.274471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 671/1383] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:44.279904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47d2c2-e9dd-4cbf-bdde-cc8fb31d005f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:44.280337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47d2c2-e9dd-4cbf-bdde-cc8fb31d005f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:44.280662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47d2c2-e9dd-4cbf-bdde-cc8fb31d005f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:44.299170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ac47d2c2-e9dd-4cbf-bdde-cc8fb31d005f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:44.300398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ac47d2c2-e9dd-4cbf-bdde-cc8fb31d005f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:44.308703 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 713/1384] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:44.351057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539bdbd-4bfd-49fd-b415-c97c954be3ef None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:44.367539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539bdbd-4bfd-49fd-b415-c97c954be3ef None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:44.373236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11c17c-75ae-4da3-bd8d-2165cccef5fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:44.374637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11c17c-75ae-4da3-bd8d-2165cccef5fe service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:44.375029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11c17c-75ae-4da3-bd8d-2165cccef5fe service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:44.375434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11c17c-75ae-4da3-bd8d-2165cccef5fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:44.384871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11c17c-75ae-4da3-bd8d-2165cccef5fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:44.386598 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11c17c-75ae-4da3-bd8d-2165cccef5fe service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:44.397080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 672/1385] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:47:46.490952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dba0b-59dc-4bed-bf35-f510e7ae9535 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:46.491449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dba0b-59dc-4bed-bf35-f510e7ae9535 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:46.491531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dba0b-59dc-4bed-bf35-f510e7ae9535 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:46.492940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 714/1386] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:46.498161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca31d32-2247-4c75-8ce9-318ba1488d37 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:46.498604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca31d32-2247-4c75-8ce9-318ba1488d37 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:46.498776 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca31d32-2247-4c75-8ce9-318ba1488d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:46.516957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3ca31d32-2247-4c75-8ce9-318ba1488d37 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:46.518151 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3ca31d32-2247-4c75-8ce9-318ba1488d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:46.526805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 673/1387] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:46.571786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dba0b-59dc-4bed-bf35-f510e7ae9535 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:46.590049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dba0b-59dc-4bed-bf35-f510e7ae9535 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:46.596552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22db941-b68f-46c8-868b-42833e0f059e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:46.597277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db941-b68f-46c8-868b-42833e0f059e service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:46.597615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db941-b68f-46c8-868b-42833e0f059e service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:46.598004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22db941-b68f-46c8-868b-42833e0f059e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:46.607280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22db941-b68f-46c8-868b-42833e0f059e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:46.609058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22db941-b68f-46c8-868b-42833e0f059e service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:46.625372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 715/1388] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:48.786807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6284f5-ff1c-45ee-9020-b552db49842b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:48.787606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6284f5-ff1c-45ee-9020-b552db49842b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:48.787855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6284f5-ff1c-45ee-9020-b552db49842b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:48.789759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 674/1389] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:48.796153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc34a63-a284-4faa-8945-db77f7b6da51 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:48.796927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc34a63-a284-4faa-8945-db77f7b6da51 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:48.796927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc34a63-a284-4faa-8945-db77f7b6da51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:48.819387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1fc34a63-a284-4faa-8945-db77f7b6da51 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:48.820722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1fc34a63-a284-4faa-8945-db77f7b6da51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:48.829610 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 716/1390] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:48.874293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6284f5-ff1c-45ee-9020-b552db49842b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:48.891979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6284f5-ff1c-45ee-9020-b552db49842b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:48.897592 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223eafee-7306-4ace-a1ac-d46821f33f71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:48.898235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eafee-7306-4ace-a1ac-d46821f33f71 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:48.898524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eafee-7306-4ace-a1ac-d46821f33f71 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:48.898806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223eafee-7306-4ace-a1ac-d46821f33f71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:48.908806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223eafee-7306-4ace-a1ac-d46821f33f71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:48.910454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223eafee-7306-4ace-a1ac-d46821f33f71 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:48.920603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 675/1391] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:51.025722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d540b-c3d9-4d81-9cce-a0d5c12d65b4 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:51.026389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d540b-c3d9-4d81-9cce-a0d5c12d65b4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:51.026389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3d540b-c3d9-4d81-9cce-a0d5c12d65b4 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:51.027637 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 717/1392] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:51.034033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225c0e2-024e-4b69-addb-359eb0d62f1a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:51.034515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225c0e2-024e-4b69-addb-359eb0d62f1a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:51.034758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225c0e2-024e-4b69-addb-359eb0d62f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:51.060363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0225c0e2-024e-4b69-addb-359eb0d62f1a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:51.061655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0225c0e2-024e-4b69-addb-359eb0d62f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:51.069694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 676/1393] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:47:51.115959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d540b-c3d9-4d81-9cce-a0d5c12d65b4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:51.130694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3d540b-c3d9-4d81-9cce-a0d5c12d65b4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:51.136223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8e70f-f5c7-4f1d-8bc3-8b470fa75ea2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:51.136906 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8e70f-f5c7-4f1d-8bc3-8b470fa75ea2 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:51.137205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8e70f-f5c7-4f1d-8bc3-8b470fa75ea2 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:51.137557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8e70f-f5c7-4f1d-8bc3-8b470fa75ea2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:51.146872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8e70f-f5c7-4f1d-8bc3-8b470fa75ea2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:51.148465 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8e70f-f5c7-4f1d-8bc3-8b470fa75ea2 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:51.159323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 718/1394] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:53.273182 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735adc9e-e163-4f77-ae96-5e4bb24ba149 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:53.273720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735adc9e-e163-4f77-ae96-5e4bb24ba149 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:53.273837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735adc9e-e163-4f77-ae96-5e4bb24ba149 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:53.275128 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 677/1395] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:53.280910 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f226e-7042-4e95-858a-647beb07f6ac None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:53.281331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f226e-7042-4e95-858a-647beb07f6ac None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:53.281495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f226e-7042-4e95-858a-647beb07f6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:53.299732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4c8f226e-7042-4e95-858a-647beb07f6ac None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:53.300974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4c8f226e-7042-4e95-858a-647beb07f6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:53.308835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 719/1396] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:53.353199 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735adc9e-e163-4f77-ae96-5e4bb24ba149 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:53.368007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735adc9e-e163-4f77-ae96-5e4bb24ba149 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:53.373676 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb42520a-5147-41a4-a035-0eba55357cac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:53.374557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42520a-5147-41a4-a035-0eba55357cac service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:53.374842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42520a-5147-41a4-a035-0eba55357cac service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:53.375128 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb42520a-5147-41a4-a035-0eba55357cac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:53.383691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42520a-5147-41a4-a035-0eba55357cac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:53.385333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb42520a-5147-41a4-a035-0eba55357cac service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:53.395731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 678/1397] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:47:54.372081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed1dea-5966-4a04-83e7-e31ff155bd4e None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:54.372650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed1dea-5966-4a04-83e7-e31ff155bd4e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:54.372650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed1dea-5966-4a04-83e7-e31ff155bd4e None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:54.374619 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 720/1398] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:54.379906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e7cb5-3873-4ae7-9067-7ae8721712fb None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:54.380359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e7cb5-3873-4ae7-9067-7ae8721712fb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:54.380555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e7cb5-3873-4ae7-9067-7ae8721712fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:54.399048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b45e7cb5-3873-4ae7-9067-7ae8721712fb None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:54.400261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b45e7cb5-3873-4ae7-9067-7ae8721712fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:54.408517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 679/1399] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:54.464164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3490c-f38c-46b5-9bc8-f8f79d71a05d None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:54.464458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3490c-f38c-46b5-9bc8-f8f79d71a05d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:54.464766 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3490c-f38c-46b5-9bc8-f8f79d71a05d None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:54.466143 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 721/1400] 10.222.0.67 () {54 vars in 962 bytes} [Thu Jun 23 11:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:54.471572 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a0ca7-c287-414f-a611-2302d36f2406 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:54.471868 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a0ca7-c287-414f-a611-2302d36f2406 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:54.472156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a0ca7-c287-414f-a611-2302d36f2406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:54.531083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-5e4a0ca7-c287-414f-a611-2302d36f2406 None None] MFA Rules not processed for user `74fdc12d728d45018ea668a4073da3b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:54.532322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5e4a0ca7-c287-414f-a611-2302d36f2406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:54.594103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 680/1401] 10.222.0.67 () {58 vars in 1059 bytes} [Thu Jun 23 11:47:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:54.601686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3490c-f38c-46b5-9bc8-f8f79d71a05d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:54.607120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=55 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:54.614537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3490c-f38c-46b5-9bc8-f8f79d71a05d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:54.618118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7058b06-93d8-43af-a8b9-23a03bfeeaf3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74fdc12d728d45018ea668a4073da3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:54.618802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7058b06-93d8-43af-a8b9-23a03bfeeaf3 service glance] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:54.619089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7058b06-93d8-43af-a8b9-23a03bfeeaf3 service glance] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:54.619367 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7058b06-93d8-43af-a8b9-23a03bfeeaf3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:54.628457 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7058b06-93d8-43af-a8b9-23a03bfeeaf3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:54.629861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7058b06-93d8-43af-a8b9-23a03bfeeaf3 service glance] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:54.639167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 722/1402] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:47:55.347937 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71239b7c-c5a6-40f0-b50b-60e54f108393 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:55.348226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71239b7c-c5a6-40f0-b50b-60e54f108393 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:55.348507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71239b7c-c5a6-40f0-b50b-60e54f108393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:55.398555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-71239b7c-c5a6-40f0-b50b-60e54f108393 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:55.399881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-71239b7c-c5a6-40f0-b50b-60e54f108393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:55.441670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:47:55.482519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 681/1403] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:47:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:55.485612 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aca57a-89e3-456f-a8c7-619ad62f465a None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:55.486240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aca57a-89e3-456f-a8c7-619ad62f465a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:55.486240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aca57a-89e3-456f-a8c7-619ad62f465a None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:55.492010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 723/1404] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:47:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:47:55.493595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51fcb9-c143-4ab8-96f0-7af3e9066883 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:55.493916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51fcb9-c143-4ab8-96f0-7af3e9066883 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:55.494252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51fcb9-c143-4ab8-96f0-7af3e9066883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:55.513421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2c51fcb9-c143-4ab8-96f0-7af3e9066883 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:55.515551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2c51fcb9-c143-4ab8-96f0-7af3e9066883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:55.517126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c71a69-e1fc-4efe-8f12-279ccf5ca3cf None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:55.517443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c71a69-e1fc-4efe-8f12-279ccf5ca3cf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:55.517847 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c71a69-e1fc-4efe-8f12-279ccf5ca3cf None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:55.519444 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 724/1405] 10.222.0.67 () {54 vars in 942 bytes} [Thu Jun 23 11:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:47:55.523706 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 682/1406] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:47:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:47:55.525581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70173232-79f6-450e-8a04-e7eb473cd20d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:55.525871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70173232-79f6-450e-8a04-e7eb473cd20d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:55.526152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70173232-79f6-450e-8a04-e7eb473cd20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:55.568693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51fcb9-c143-4ab8-96f0-7af3e9066883 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:55.575984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-70173232-79f6-450e-8a04-e7eb473cd20d None None] MFA Rules not processed for user `eb9b9af5e0b44e34837e1cdb5ff6d33e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:47:55.578318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-70173232-79f6-450e-8a04-e7eb473cd20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:47:55.581909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51fcb9-c143-4ab8-96f0-7af3e9066883 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:55.588125 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e261f38-1e43-4f7a-af6d-e4bd677439d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:55.588694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e261f38-1e43-4f7a-af6d-e4bd677439d5 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:55.589053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e261f38-1e43-4f7a-af6d-e4bd677439d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:55.589405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e261f38-1e43-4f7a-af6d-e4bd677439d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:55.598192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e261f38-1e43-4f7a-af6d-e4bd677439d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:55.599853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e261f38-1e43-4f7a-af6d-e4bd677439d5 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:55.610139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 683/1407] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:47:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:47:55.651264 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 725/1408] 10.222.0.67 () {58 vars in 1039 bytes} [Thu Jun 23 11:47:55 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:47:55.660847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e261f38-1e43-4f7a-af6d-e4bd677439d5 service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:47:55.675641 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e261f38-1e43-4f7a-af6d-e4bd677439d5 service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:47:55.680954 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7b8f4-f6a1-4e32-a417-56e95e53d883 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:47:55.681648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7b8f4-f6a1-4e32-a417-56e95e53d883 service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:47:55.681939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7b8f4-f6a1-4e32-a417-56e95e53d883 service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:47:55.682224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7b8f4-f6a1-4e32-a417-56e95e53d883 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:47:55.693402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7b8f4-f6a1-4e32-a417-56e95e53d883 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:47:55.695653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7b8f4-f6a1-4e32-a417-56e95e53d883 service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:47:55.705689 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 684/1409] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:47:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:48:05.648399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=49 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:48:05.701777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=24 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:48:18.001934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bcb80-f6dd-4504-8af8-f8c209a9cc1b None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:18.002641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bcb80-f6dd-4504-8af8-f8c209a9cc1b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:18.002641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bcb80-f6dd-4504-8af8-f8c209a9cc1b None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:18.003886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 726/1410] 10.222.0.67 () {54 vars in 918 bytes} [Thu Jun 23 11:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:18.009056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b271b70-1599-495b-b2fa-d88904fae708 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:18.009539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b271b70-1599-495b-b2fa-d88904fae708 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:18.009648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b271b70-1599-495b-b2fa-d88904fae708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:18.027749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3b271b70-1599-495b-b2fa-d88904fae708 None None] MFA Rules not processed for user `ee971347d23040d3984f961025e8157e`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:18.028978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3b271b70-1599-495b-b2fa-d88904fae708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:18.038510 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 685/1411] 10.222.0.67 () {58 vars in 1015 bytes} [Thu Jun 23 11:48:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:18.052553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1694ad8-0998-4d45-beae-51fddfe58fb2 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:18.052880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1694ad8-0998-4d45-beae-51fddfe58fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:18.053176 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1694ad8-0998-4d45-beae-51fddfe58fb2 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:18.054533 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 727/1412] 10.222.0.67 () {54 vars in 918 bytes} [Thu Jun 23 11:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:18.059096 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f337af-1ad6-41b7-8a37-8ed50b91e53d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:18.059385 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f337af-1ad6-41b7-8a37-8ed50b91e53d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:18.059665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f337af-1ad6-41b7-8a37-8ed50b91e53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:18.078168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-26f337af-1ad6-41b7-8a37-8ed50b91e53d None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:18.079357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-26f337af-1ad6-41b7-8a37-8ed50b91e53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:18.088356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 686/1413] 10.222.0.67 () {58 vars in 1015 bytes} [Thu Jun 23 11:48:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:48:20.633588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c9516-8e7e-4a5a-b9ff-48efcb903652 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:20.634168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c9516-8e7e-4a5a-b9ff-48efcb903652 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:20.634168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c9516-8e7e-4a5a-b9ff-48efcb903652 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:20.635530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 728/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:20.640840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7fde0-8549-4366-a97b-684cc7f9f9f7 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:20.641263 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7fde0-8549-4366-a97b-684cc7f9f9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:20.641394 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7fde0-8549-4366-a97b-684cc7f9f9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:20.658881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-49e7fde0-8549-4366-a97b-684cc7f9f9f7 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:20.660084 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-49e7fde0-8549-4366-a97b-684cc7f9f9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:20.668373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 687/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:48:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:20.712237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c9516-8e7e-4a5a-b9ff-48efcb903652 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:20.727301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c9516-8e7e-4a5a-b9ff-48efcb903652 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:20.732887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3024467-1c28-4ec4-a2a6-86b51b40c2b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:20.733524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3024467-1c28-4ec4-a2a6-86b51b40c2b1 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:20.733820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3024467-1c28-4ec4-a2a6-86b51b40c2b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:20.734625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3024467-1c28-4ec4-a2a6-86b51b40c2b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:20.743046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3024467-1c28-4ec4-a2a6-86b51b40c2b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:20.744630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3024467-1c28-4ec4-a2a6-86b51b40c2b1 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:20.754243 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 729/1416] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:21.653222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3dafd-baf7-4772-af6f-3aed50e9faa7 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:21.653791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3dafd-baf7-4772-af6f-3aed50e9faa7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:21.653900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3dafd-baf7-4772-af6f-3aed50e9faa7 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:21.655214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 688/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:21.660354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cbb25-77ca-435c-9598-ce3844ec0d56 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:21.660862 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cbb25-77ca-435c-9598-ce3844ec0d56 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:21.660970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cbb25-77ca-435c-9598-ce3844ec0d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:21.679565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-fa0cbb25-77ca-435c-9598-ce3844ec0d56 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:21.680812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fa0cbb25-77ca-435c-9598-ce3844ec0d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:21.689356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 730/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:21.836652 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3dafd-baf7-4772-af6f-3aed50e9faa7 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:21.851265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3dafd-baf7-4772-af6f-3aed50e9faa7 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:21.858247 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d1471-f9da-4f21-8929-0e220bf4d450 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:21.859065 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d1471-f9da-4f21-8929-0e220bf4d450 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:21.859409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d1471-f9da-4f21-8929-0e220bf4d450 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:21.859667 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d1471-f9da-4f21-8929-0e220bf4d450 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:21.868480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d1471-f9da-4f21-8929-0e220bf4d450 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:21.870114 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d1471-f9da-4f21-8929-0e220bf4d450 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:21.880682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 689/1419] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:22.770303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccde67-380d-4294-97b4-7c3106ae4860 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:22.770861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccde67-380d-4294-97b4-7c3106ae4860 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:22.770969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccde67-380d-4294-97b4-7c3106ae4860 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:22.772328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 731/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:22.777450 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a0037-2861-478e-9a27-8b75910a6d7a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:22.777874 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a0037-2861-478e-9a27-8b75910a6d7a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:22.778019 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a0037-2861-478e-9a27-8b75910a6d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:22.796439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-222a0037-2861-478e-9a27-8b75910a6d7a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:22.797753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-222a0037-2861-478e-9a27-8b75910a6d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:22.805547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 690/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:22.957077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccde67-380d-4294-97b4-7c3106ae4860 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:22.971170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccde67-380d-4294-97b4-7c3106ae4860 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:22.976575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e2b2d-6b50-463f-937a-60c8c9156b25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:22.977236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e2b2d-6b50-463f-937a-60c8c9156b25 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:22.977521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e2b2d-6b50-463f-937a-60c8c9156b25 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:22.977797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e2b2d-6b50-463f-937a-60c8c9156b25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:22.985898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e2b2d-6b50-463f-937a-60c8c9156b25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:22.987464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e2b2d-6b50-463f-937a-60c8c9156b25 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:22.996779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 732/1422] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:23.848549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fccc89-4791-4061-8fce-02837e8e4ff9 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:23.848999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fccc89-4791-4061-8fce-02837e8e4ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:23.849137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fccc89-4791-4061-8fce-02837e8e4ff9 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:23.850523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 691/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:23.855557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311dbd0-adce-4f89-ad1a-a8859c1f2f30 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:23.856011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311dbd0-adce-4f89-ad1a-a8859c1f2f30 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:23.856152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311dbd0-adce-4f89-ad1a-a8859c1f2f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:23.874702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9311dbd0-adce-4f89-ad1a-a8859c1f2f30 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:23.875935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9311dbd0-adce-4f89-ad1a-a8859c1f2f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:23.883896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 733/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:24.033353 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fccc89-4791-4061-8fce-02837e8e4ff9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:24.048739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fccc89-4791-4061-8fce-02837e8e4ff9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:24.054086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d40d62-2a9b-46f4-b50b-44c8add311f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:24.054718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d40d62-2a9b-46f4-b50b-44c8add311f8 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:24.055001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d40d62-2a9b-46f4-b50b-44c8add311f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:24.055273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d40d62-2a9b-46f4-b50b-44c8add311f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:24.063477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d40d62-2a9b-46f4-b50b-44c8add311f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:24.065082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d40d62-2a9b-46f4-b50b-44c8add311f8 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:24.074356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 692/1425] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:24.945797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f2abd-8df4-4210-9cf5-5d8b0de45278 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:24.946334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f2abd-8df4-4210-9cf5-5d8b0de45278 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:24.946334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f2abd-8df4-4210-9cf5-5d8b0de45278 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:24.947661 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 734/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:24.953264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd68eb8-cb02-48b4-8d30-bdfb0a81e905 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:24.953700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd68eb8-cb02-48b4-8d30-bdfb0a81e905 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:24.953843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd68eb8-cb02-48b4-8d30-bdfb0a81e905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:24.972990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-4dd68eb8-cb02-48b4-8d30-bdfb0a81e905 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:24.974198 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4dd68eb8-cb02-48b4-8d30-bdfb0a81e905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:24.981949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 693/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:48:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:25.131946 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f2abd-8df4-4210-9cf5-5d8b0de45278 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:25.146927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f2abd-8df4-4210-9cf5-5d8b0de45278 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:25.152470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8958e2b-eea8-47ac-941d-c2254582feb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:25.153140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8958e2b-eea8-47ac-941d-c2254582feb4 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:25.153431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8958e2b-eea8-47ac-941d-c2254582feb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:25.153714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8958e2b-eea8-47ac-941d-c2254582feb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:25.162941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8958e2b-eea8-47ac-941d-c2254582feb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:25.164524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8958e2b-eea8-47ac-941d-c2254582feb4 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:25.174102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 735/1428] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:25.988267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce1e32-411c-4850-9a23-7733c65ff548 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:25.988855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce1e32-411c-4850-9a23-7733c65ff548 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:25.988855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ce1e32-411c-4850-9a23-7733c65ff548 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:25.990410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 694/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:25.995597 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150b11c-2806-4911-8d23-52f933493e15 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:25.996140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150b11c-2806-4911-8d23-52f933493e15 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:25.996433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150b11c-2806-4911-8d23-52f933493e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:26.014502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f150b11c-2806-4911-8d23-52f933493e15 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:26.015708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f150b11c-2806-4911-8d23-52f933493e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:26.023480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 736/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:48:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:26.174058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce1e32-411c-4850-9a23-7733c65ff548 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:26.188372 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ce1e32-411c-4850-9a23-7733c65ff548 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:26.194053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b0b5a-4e46-46a3-9102-96d6f07f323e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:26.194687 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b0b5a-4e46-46a3-9102-96d6f07f323e service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:26.195053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b0b5a-4e46-46a3-9102-96d6f07f323e service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:26.195336 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b0b5a-4e46-46a3-9102-96d6f07f323e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:26.203393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b0b5a-4e46-46a3-9102-96d6f07f323e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:26.205029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b0b5a-4e46-46a3-9102-96d6f07f323e service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:26.214147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 695/1431] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:27.024795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51266a3-87a1-4755-b243-429b1eb69edf None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:27.025361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51266a3-87a1-4755-b243-429b1eb69edf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:27.025361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51266a3-87a1-4755-b243-429b1eb69edf None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:27.026718 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 737/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:27.031631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376074e-06cd-4b81-95b8-d25adfa89b3c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:27.032039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376074e-06cd-4b81-95b8-d25adfa89b3c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:27.032202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376074e-06cd-4b81-95b8-d25adfa89b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:27.050000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d376074e-06cd-4b81-95b8-d25adfa89b3c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:27.051209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d376074e-06cd-4b81-95b8-d25adfa89b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:27.059069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 696/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:27.209460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51266a3-87a1-4755-b243-429b1eb69edf None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:27.223849 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51266a3-87a1-4755-b243-429b1eb69edf None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:27.229304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c4c51-51c1-4748-88f9-41429ff2d1e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:27.229964 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c4c51-51c1-4748-88f9-41429ff2d1e4 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:27.230236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c4c51-51c1-4748-88f9-41429ff2d1e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:27.230515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c4c51-51c1-4748-88f9-41429ff2d1e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:27.239746 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c4c51-51c1-4748-88f9-41429ff2d1e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:27.241393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c4c51-51c1-4748-88f9-41429ff2d1e4 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:27.251685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 738/1434] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:28.053535 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361b268-c188-44c2-a804-690715738576 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:28.054099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361b268-c188-44c2-a804-690715738576 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:28.054099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361b268-c188-44c2-a804-690715738576 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:28.055480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 697/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:28.060469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909cbd9-123a-4617-9020-595fc90bf700 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:28.060974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909cbd9-123a-4617-9020-595fc90bf700 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:28.061074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909cbd9-123a-4617-9020-595fc90bf700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:28.079170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7909cbd9-123a-4617-9020-595fc90bf700 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:28.080374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7909cbd9-123a-4617-9020-595fc90bf700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:28.088185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 739/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:28.238184 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361b268-c188-44c2-a804-690715738576 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:28.254196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361b268-c188-44c2-a804-690715738576 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:28.259524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44adf85-57e9-470e-bf6a-c90bfdc85d10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:28.260158 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44adf85-57e9-470e-bf6a-c90bfdc85d10 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:28.260444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44adf85-57e9-470e-bf6a-c90bfdc85d10 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:28.260772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44adf85-57e9-470e-bf6a-c90bfdc85d10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:28.269384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44adf85-57e9-470e-bf6a-c90bfdc85d10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:28.270959 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44adf85-57e9-470e-bf6a-c90bfdc85d10 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:28.280167 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 698/1437] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:29.089528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87646206-45b9-4be8-9275-054b478d7ff7 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:29.090075 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87646206-45b9-4be8-9275-054b478d7ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:29.090075 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87646206-45b9-4be8-9275-054b478d7ff7 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:29.091456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 740/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:29.097016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7524ab0-02f9-45ad-b800-68ad5e12197d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:29.097411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7524ab0-02f9-45ad-b800-68ad5e12197d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:29.097615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7524ab0-02f9-45ad-b800-68ad5e12197d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:29.116226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b7524ab0-02f9-45ad-b800-68ad5e12197d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:29.117474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b7524ab0-02f9-45ad-b800-68ad5e12197d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:29.125285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 699/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:29.274342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87646206-45b9-4be8-9275-054b478d7ff7 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:29.289214 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87646206-45b9-4be8-9275-054b478d7ff7 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:29.294501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dbeb70-6772-4a90-8a70-11cdbbd26be1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:29.295131 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbeb70-6772-4a90-8a70-11cdbbd26be1 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:29.296039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbeb70-6772-4a90-8a70-11cdbbd26be1 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:29.296320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dbeb70-6772-4a90-8a70-11cdbbd26be1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:29.304554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbeb70-6772-4a90-8a70-11cdbbd26be1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:29.306155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dbeb70-6772-4a90-8a70-11cdbbd26be1 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:29.315648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 741/1440] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:31.265727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a0261-6bcd-4829-8536-ab0fc2a92dfb None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:31.266286 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a0261-6bcd-4829-8536-ab0fc2a92dfb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:31.266286 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005a0261-6bcd-4829-8536-ab0fc2a92dfb None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:31.267594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 700/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:48:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:31.272895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7e4c6-623b-4141-bb8d-98d5616617b2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:31.273350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7e4c6-623b-4141-bb8d-98d5616617b2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:31.273471 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7e4c6-623b-4141-bb8d-98d5616617b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:31.291110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8dc7e4c6-623b-4141-bb8d-98d5616617b2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:31.292312 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8dc7e4c6-623b-4141-bb8d-98d5616617b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:31.300051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 742/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:31.338583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a0261-6bcd-4829-8536-ab0fc2a92dfb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:31.349011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005a0261-6bcd-4829-8536-ab0fc2a92dfb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:31.352432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1754dc-1721-4304-8acf-311b11d5c130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:31.353091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1754dc-1721-4304-8acf-311b11d5c130 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:31.353374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1754dc-1721-4304-8acf-311b11d5c130 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:31.353649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1754dc-1721-4304-8acf-311b11d5c130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:31.362415 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1754dc-1721-4304-8acf-311b11d5c130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:31.363824 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1754dc-1721-4304-8acf-311b11d5c130 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:31.374052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 701/1443] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:33.568617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed410afa-5be4-4eda-94fa-d0d387833f89 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:33.569196 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed410afa-5be4-4eda-94fa-d0d387833f89 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:33.569196 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed410afa-5be4-4eda-94fa-d0d387833f89 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:33.570589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 743/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:33.575972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d0006-a757-47e7-85bb-e666027a8ab1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:33.576435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d0006-a757-47e7-85bb-e666027a8ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:33.576548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d0006-a757-47e7-85bb-e666027a8ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:33.597812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b18d0006-a757-47e7-85bb-e666027a8ab1 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:33.599038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b18d0006-a757-47e7-85bb-e666027a8ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:33.607019 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 702/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:48:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:33.645151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed410afa-5be4-4eda-94fa-d0d387833f89 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:33.656506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed410afa-5be4-4eda-94fa-d0d387833f89 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:33.660173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b021a72-497a-4337-a0f9-56c72f379dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:33.660879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b021a72-497a-4337-a0f9-56c72f379dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:33.661169 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b021a72-497a-4337-a0f9-56c72f379dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:33.661453 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b021a72-497a-4337-a0f9-56c72f379dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:33.671744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b021a72-497a-4337-a0f9-56c72f379dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:33.673226 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b021a72-497a-4337-a0f9-56c72f379dd4 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:33.682660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 744/1446] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:36.027665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496cbec-4f4a-43ee-ab45-ab464cd2c4f7 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:36.028204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496cbec-4f4a-43ee-ab45-ab464cd2c4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:36.028204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496cbec-4f4a-43ee-ab45-ab464cd2c4f7 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:36.030172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 703/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:48:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:36.035294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb80871-1365-4980-a4ff-290248733be3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:36.035706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb80871-1365-4980-a4ff-290248733be3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:36.035863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb80871-1365-4980-a4ff-290248733be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:36.055259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ffb80871-1365-4980-a4ff-290248733be3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:36.056502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ffb80871-1365-4980-a4ff-290248733be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:36.065086 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 745/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:36.104394 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496cbec-4f4a-43ee-ab45-ab464cd2c4f7 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:36.115135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496cbec-4f4a-43ee-ab45-ab464cd2c4f7 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:36.118958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf4190-0f46-4cdb-b8e0-dc72dda0198f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:36.119560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf4190-0f46-4cdb-b8e0-dc72dda0198f service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:36.119852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf4190-0f46-4cdb-b8e0-dc72dda0198f service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:36.120141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf4190-0f46-4cdb-b8e0-dc72dda0198f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:36.129951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf4190-0f46-4cdb-b8e0-dc72dda0198f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:36.131400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf4190-0f46-4cdb-b8e0-dc72dda0198f service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:36.141509 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 704/1449] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:38.309787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a343d3-eadd-44e8-a05d-051eee988965 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:38.310356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a343d3-eadd-44e8-a05d-051eee988965 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:38.310356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a343d3-eadd-44e8-a05d-051eee988965 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:38.312375 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 746/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:48:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:38.317754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc8f16-2240-4d3e-95f6-b9a3ffe4708d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:38.318184 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc8f16-2240-4d3e-95f6-b9a3ffe4708d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:38.318321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc8f16-2240-4d3e-95f6-b9a3ffe4708d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:38.335943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-73cc8f16-2240-4d3e-95f6-b9a3ffe4708d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:38.337196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-73cc8f16-2240-4d3e-95f6-b9a3ffe4708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:38.345038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 705/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:38.385498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a343d3-eadd-44e8-a05d-051eee988965 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:38.387191 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e7a343d3-eadd-44e8-a05d-051eee988965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:38.399988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a343d3-eadd-44e8-a05d-051eee988965 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:38.403498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca9e97-f1a4-42cd-a961-cf7e542a556b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:38.404129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca9e97-f1a4-42cd-a961-cf7e542a556b service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:38.404417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca9e97-f1a4-42cd-a961-cf7e542a556b service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:38.404726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca9e97-f1a4-42cd-a961-cf7e542a556b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:38.416070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca9e97-f1a4-42cd-a961-cf7e542a556b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:38.417781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca9e97-f1a4-42cd-a961-cf7e542a556b service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:38.428015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 747/1452] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:40.605934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d53e9f-4bed-4c3e-a28e-12a421c78795 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:40.606504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d53e9f-4bed-4c3e-a28e-12a421c78795 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:40.606504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d53e9f-4bed-4c3e-a28e-12a421c78795 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:40.608389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 706/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:40.613581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ee945-ea5e-47b0-9bea-fecc73a12686 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:40.613996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ee945-ea5e-47b0-9bea-fecc73a12686 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:40.614155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ee945-ea5e-47b0-9bea-fecc73a12686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:40.632192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-378ee945-ea5e-47b0-9bea-fecc73a12686 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:40.633434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-378ee945-ea5e-47b0-9bea-fecc73a12686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:40.641681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 748/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:48:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:40.679968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d53e9f-4bed-4c3e-a28e-12a421c78795 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:40.692004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d53e9f-4bed-4c3e-a28e-12a421c78795 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:40.695748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32400f-af74-4349-9887-a65c0513815b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:40.696390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32400f-af74-4349-9887-a65c0513815b service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:40.696705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32400f-af74-4349-9887-a65c0513815b service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:40.696991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32400f-af74-4349-9887-a65c0513815b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:40.706162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32400f-af74-4349-9887-a65c0513815b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:40.707582 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32400f-af74-4349-9887-a65c0513815b service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:40.716970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 707/1455] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:42.819142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429f9e3-78f4-4377-81c5-bd1c613c86c0 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:42.819708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429f9e3-78f4-4377-81c5-bd1c613c86c0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:42.819708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3429f9e3-78f4-4377-81c5-bd1c613c86c0 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:42.821112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 749/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:42.826875 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e589ff-31d4-49f8-bdae-6c29b17946ef None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:42.827285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e589ff-31d4-49f8-bdae-6c29b17946ef None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:42.827444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e589ff-31d4-49f8-bdae-6c29b17946ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:42.845377 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-27e589ff-31d4-49f8-bdae-6c29b17946ef None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:42.846593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-27e589ff-31d4-49f8-bdae-6c29b17946ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:42.854423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 708/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:48:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:42.891987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429f9e3-78f4-4377-81c5-bd1c613c86c0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:42.902635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3429f9e3-78f4-4377-81c5-bd1c613c86c0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:42.906777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d8235-ef3c-44fb-abde-e3c76ad95b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:42.907412 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d8235-ef3c-44fb-abde-e3c76ad95b7c service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:42.907696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d8235-ef3c-44fb-abde-e3c76ad95b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:42.907974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d8235-ef3c-44fb-abde-e3c76ad95b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:42.917316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d8235-ef3c-44fb-abde-e3c76ad95b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:42.918792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d8235-ef3c-44fb-abde-e3c76ad95b7c service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:42.927987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 750/1458] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:45.027217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb75ba5-ba46-4ca5-8ae0-8252443a4408 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:45.027792 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb75ba5-ba46-4ca5-8ae0-8252443a4408 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:45.027792 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb75ba5-ba46-4ca5-8ae0-8252443a4408 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:45.029204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 709/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:45.034481 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b76b3-5dcf-4e42-b948-3dbb35e3ed41 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:45.034891 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b76b3-5dcf-4e42-b948-3dbb35e3ed41 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:45.035050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b76b3-5dcf-4e42-b948-3dbb35e3ed41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:45.053310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5e1b76b3-5dcf-4e42-b948-3dbb35e3ed41 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:45.054516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5e1b76b3-5dcf-4e42-b948-3dbb35e3ed41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:45.063137 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 751/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:45.099861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb75ba5-ba46-4ca5-8ae0-8252443a4408 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:45.111222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb75ba5-ba46-4ca5-8ae0-8252443a4408 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:45.114831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6a6615-9974-45a5-8561-df3e2499694c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:45.115466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a6615-9974-45a5-8561-df3e2499694c service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:45.115757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a6615-9974-45a5-8561-df3e2499694c service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:45.116040 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6a6615-9974-45a5-8561-df3e2499694c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:45.125590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a6615-9974-45a5-8561-df3e2499694c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:45.127260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6a6615-9974-45a5-8561-df3e2499694c service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:45.136597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 710/1461] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:47.247078 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a1316-3783-4b0b-ac7f-4bfa056eb664 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:47.247660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a1316-3783-4b0b-ac7f-4bfa056eb664 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:47.247660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a1316-3783-4b0b-ac7f-4bfa056eb664 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:47.249599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 752/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:47.254989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c39e1c-78dc-4e05-bba7-364323911bc0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:47.255408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c39e1c-78dc-4e05-bba7-364323911bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:47.255547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c39e1c-78dc-4e05-bba7-364323911bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:47.261627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=79 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:48:47.274710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b1c39e1c-78dc-4e05-bba7-364323911bc0 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:47.275923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b1c39e1c-78dc-4e05-bba7-364323911bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:47.283999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 711/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:47.322635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a1316-3783-4b0b-ac7f-4bfa056eb664 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:47.328051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=73 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:48:47.334764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a1316-3783-4b0b-ac7f-4bfa056eb664 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:47.338340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f4b1c-15d0-460b-b64b-4f4c92e33618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:47.338979 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4b1c-15d0-460b-b64b-4f4c92e33618 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:47.339266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4b1c-15d0-460b-b64b-4f4c92e33618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:47.339544 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f4b1c-15d0-460b-b64b-4f4c92e33618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:47.348453 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f4b1c-15d0-460b-b64b-4f4c92e33618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:47.349920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f4b1c-15d0-460b-b64b-4f4c92e33618 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:47.360386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 753/1464] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:49.431625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68a446-7462-4285-8512-050db103308b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:49.432196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68a446-7462-4285-8512-050db103308b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:49.432196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68a446-7462-4285-8512-050db103308b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:49.433643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 712/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:49.438982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04973768-26dc-4d00-a731-064fd5d0ed5f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:49.439431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04973768-26dc-4d00-a731-064fd5d0ed5f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:49.439538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04973768-26dc-4d00-a731-064fd5d0ed5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:49.456994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-04973768-26dc-4d00-a731-064fd5d0ed5f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:49.458205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-04973768-26dc-4d00-a731-064fd5d0ed5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:49.466131 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 754/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:49.503042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68a446-7462-4285-8512-050db103308b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:49.513606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68a446-7462-4285-8512-050db103308b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:49.517166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69a7cc8-c560-4dc0-bedd-4c2151a5590e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:49.517825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a7cc8-c560-4dc0-bedd-4c2151a5590e service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:49.518107 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a7cc8-c560-4dc0-bedd-4c2151a5590e service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:49.518393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69a7cc8-c560-4dc0-bedd-4c2151a5590e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:49.527235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a7cc8-c560-4dc0-bedd-4c2151a5590e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:49.528649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69a7cc8-c560-4dc0-bedd-4c2151a5590e service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:49.538763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 713/1467] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:48:51.583259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d3480-8ff5-414f-bc86-94a496d314b8 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:51.583826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d3480-8ff5-414f-bc86-94a496d314b8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:51.583826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d3480-8ff5-414f-bc86-94a496d314b8 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:51.585253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 755/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:48:51.590653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7ee6f-0f37-4564-884a-516b6010164e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:51.591055 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7ee6f-0f37-4564-884a-516b6010164e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:51.591249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7ee6f-0f37-4564-884a-516b6010164e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:51.608499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-66e7ee6f-0f37-4564-884a-516b6010164e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:48:51.609734 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-66e7ee6f-0f37-4564-884a-516b6010164e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:48:51.617570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 714/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:48:51.654140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d3480-8ff5-414f-bc86-94a496d314b8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:48:51.665198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d3480-8ff5-414f-bc86-94a496d314b8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:48:51.668782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceec59f-4575-4337-81a4-26b6dadb8f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:48:51.669421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceec59f-4575-4337-81a4-26b6dadb8f31 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:48:51.669712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceec59f-4575-4337-81a4-26b6dadb8f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:48:51.669999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceec59f-4575-4337-81a4-26b6dadb8f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:48:51.679587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceec59f-4575-4337-81a4-26b6dadb8f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:48:51.681076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceec59f-4575-4337-81a4-26b6dadb8f31 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:48:51.691735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 756/1470] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:48:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:01.603499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=11 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:49:01.687259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=14 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:49:11.726157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d92d9-9c62-48dc-8fb1-0bc37567e7f3 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:11.726734 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d92d9-9c62-48dc-8fb1-0bc37567e7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:11.726734 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d92d9-9c62-48dc-8fb1-0bc37567e7f3 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:11.728201 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 715/1471] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:49:11.733801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423059b0-fcc6-4351-bbf7-99543b50a05e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:11.734244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423059b0-fcc6-4351-bbf7-99543b50a05e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:11.734361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423059b0-fcc6-4351-bbf7-99543b50a05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:11.753631 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-423059b0-fcc6-4351-bbf7-99543b50a05e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:49:11.755035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-423059b0-fcc6-4351-bbf7-99543b50a05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:11.763332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 757/1472] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:49:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:49:11.812210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547df6ff-6bd6-4794-b1de-e6fba3792ed0 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:11.812503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547df6ff-6bd6-4794-b1de-e6fba3792ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:11.812850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547df6ff-6bd6-4794-b1de-e6fba3792ed0 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:11.814212 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 716/1473] 10.222.0.67 () {54 vars in 962 bytes} [Thu Jun 23 11:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:49:11.819158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbc21f-a679-4e47-b9aa-69f9184d8111 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:11.819460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbc21f-a679-4e47-b9aa-69f9184d8111 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:11.819747 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbc21f-a679-4e47-b9aa-69f9184d8111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:11.838648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9dfbc21f-a679-4e47-b9aa-69f9184d8111 None None] MFA Rules not processed for user `74fdc12d728d45018ea668a4073da3b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:49:11.839853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9dfbc21f-a679-4e47-b9aa-69f9184d8111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:11.846295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 758/1474] 10.222.0.67 () {58 vars in 1059 bytes} [Thu Jun 23 11:49:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:49:11.853773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547df6ff-6bd6-4794-b1de-e6fba3792ed0 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:11.868322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547df6ff-6bd6-4794-b1de-e6fba3792ed0 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:11.872949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac43ac-0f07-401d-9754-dea4107dca48 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74fdc12d728d45018ea668a4073da3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:11.874225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac43ac-0f07-401d-9754-dea4107dca48 service glance] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:11.874615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac43ac-0f07-401d-9754-dea4107dca48 service glance] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:11.874895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac43ac-0f07-401d-9754-dea4107dca48 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:11.884432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac43ac-0f07-401d-9754-dea4107dca48 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:11.885945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac43ac-0f07-401d-9754-dea4107dca48 service glance] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:11.895990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 717/1475] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:49:14.044734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece100f-e536-40ae-a14f-6f0d02044531 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:14.045309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece100f-e536-40ae-a14f-6f0d02044531 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:14.045408 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ece100f-e536-40ae-a14f-6f0d02044531 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:14.047247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 759/1476] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:49:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:49:14.052374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b9971-18f5-4325-a102-b770abbd7892 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:14.052788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b9971-18f5-4325-a102-b770abbd7892 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:14.052985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b9971-18f5-4325-a102-b770abbd7892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:14.070602 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-563b9971-18f5-4325-a102-b770abbd7892 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:49:14.071812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-563b9971-18f5-4325-a102-b770abbd7892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:14.079746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 718/1477] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:49:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:49:14.103524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece100f-e536-40ae-a14f-6f0d02044531 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:14.116518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ece100f-e536-40ae-a14f-6f0d02044531 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:14.121091 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c630dbe-d638-432c-87ae-3b2d7018e0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:14.121738 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c630dbe-d638-432c-87ae-3b2d7018e0dd admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:14.122026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c630dbe-d638-432c-87ae-3b2d7018e0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:14.122306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c630dbe-d638-432c-87ae-3b2d7018e0dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:14.123068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c630dbe-d638-432c-87ae-3b2d7018e0dd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:14.124915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-9c630dbe-d638-432c-87ae-3b2d7018e0dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:49:14.124915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jun 23 11:49:14.124915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: Jun 23 11:49:14.126062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c630dbe-d638-432c-87ae-3b2d7018e0dd admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:14.262054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 760/1478] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 23 11:49:14 2022] GET /identity/v3/projects => generated 2553 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:49:16.343269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef01876-6d7a-4f20-825f-572ab46d2728 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:16.343851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef01876-6d7a-4f20-825f-572ab46d2728 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:16.343851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef01876-6d7a-4f20-825f-572ab46d2728 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:16.345257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 719/1479] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:49:16.350565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e420a0-514e-4dd0-87d3-49f44220d5a9 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:16.351028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e420a0-514e-4dd0-87d3-49f44220d5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:16.351139 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e420a0-514e-4dd0-87d3-49f44220d5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:16.368829 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-90e420a0-514e-4dd0-87d3-49f44220d5a9 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:49:16.369959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-90e420a0-514e-4dd0-87d3-49f44220d5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:16.378347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 761/1480] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:49:16.421699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef01876-6d7a-4f20-825f-572ab46d2728 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:16.439653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef01876-6d7a-4f20-825f-572ab46d2728 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:16.445159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fa6db-0230-4f3a-b58d-b4005eade4f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:16.445814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fa6db-0230-4f3a-b58d-b4005eade4f2 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:16.446138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fa6db-0230-4f3a-b58d-b4005eade4f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:16.446425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fa6db-0230-4f3a-b58d-b4005eade4f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:16.455434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fa6db-0230-4f3a-b58d-b4005eade4f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:16.457241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fa6db-0230-4f3a-b58d-b4005eade4f2 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:16.466718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 720/1481] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:18.748031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653cd38-cb7c-4f49-a61b-c062d00bedca None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:18.748605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653cd38-cb7c-4f49-a61b-c062d00bedca None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:18.748605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653cd38-cb7c-4f49-a61b-c062d00bedca None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:18.750014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 762/1482] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:49:18.755785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10118d83-f5c8-4451-b18d-0bcf8f6c031f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:18.756285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10118d83-f5c8-4451-b18d-0bcf8f6c031f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:18.756394 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10118d83-f5c8-4451-b18d-0bcf8f6c031f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:18.774711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-10118d83-f5c8-4451-b18d-0bcf8f6c031f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:49:18.775991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-10118d83-f5c8-4451-b18d-0bcf8f6c031f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:18.785261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 721/1483] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:49:18.823513 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653cd38-cb7c-4f49-a61b-c062d00bedca None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:18.835010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653cd38-cb7c-4f49-a61b-c062d00bedca None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:18.838855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922673b-23ed-49ab-a51e-ba14acc848db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:18.839538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922673b-23ed-49ab-a51e-ba14acc848db service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:18.839820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922673b-23ed-49ab-a51e-ba14acc848db service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:18.840096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922673b-23ed-49ab-a51e-ba14acc848db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:18.850593 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922673b-23ed-49ab-a51e-ba14acc848db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:18.852026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922673b-23ed-49ab-a51e-ba14acc848db service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:18.862051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 763/1484] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:20.980568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611dc8b-eefd-4ed2-aca3-21b196d5879a None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:20.981127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611dc8b-eefd-4ed2-aca3-21b196d5879a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:20.981331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611dc8b-eefd-4ed2-aca3-21b196d5879a None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:20.982572 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 722/1485] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:49:20.988001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3257435-a1fd-47da-9452-303f47964650 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:20.988408 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3257435-a1fd-47da-9452-303f47964650 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:20.988560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3257435-a1fd-47da-9452-303f47964650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:21.006549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a3257435-a1fd-47da-9452-303f47964650 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:49:21.007742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a3257435-a1fd-47da-9452-303f47964650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:21.016063 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 764/1486] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:49:21.053790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611dc8b-eefd-4ed2-aca3-21b196d5879a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:21.065292 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611dc8b-eefd-4ed2-aca3-21b196d5879a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:21.068962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd10ac-fa53-4948-9d79-8d15285817a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:21.069593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd10ac-fa53-4948-9d79-8d15285817a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:21.069879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd10ac-fa53-4948-9d79-8d15285817a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:21.070165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd10ac-fa53-4948-9d79-8d15285817a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:21.079069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd10ac-fa53-4948-9d79-8d15285817a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:21.080486 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd10ac-fa53-4948-9d79-8d15285817a8 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:21.090119 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 723/1487] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:23.303058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc5b39-a836-4e3e-b49b-d97ed3aa6da1 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:23.303635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc5b39-a836-4e3e-b49b-d97ed3aa6da1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:23.303635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc5b39-a836-4e3e-b49b-d97ed3aa6da1 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:23.305403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 765/1488] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:49:23.311141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfb247-3756-4f9b-98a1-e2a17fddbe49 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:23.311646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfb247-3756-4f9b-98a1-e2a17fddbe49 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:23.311743 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfb247-3756-4f9b-98a1-e2a17fddbe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:23.329886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8ebfb247-3756-4f9b-98a1-e2a17fddbe49 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:49:23.331178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8ebfb247-3756-4f9b-98a1-e2a17fddbe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:23.339016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 724/1489] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:49:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:49:23.380406 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc5b39-a836-4e3e-b49b-d97ed3aa6da1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:23.392057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc5b39-a836-4e3e-b49b-d97ed3aa6da1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:23.399291 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcabc3-90f9-43a6-b7e2-28fbf65aaf68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:23.399951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcabc3-90f9-43a6-b7e2-28fbf65aaf68 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:23.400234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcabc3-90f9-43a6-b7e2-28fbf65aaf68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:23.400505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcabc3-90f9-43a6-b7e2-28fbf65aaf68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:23.410973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcabc3-90f9-43a6-b7e2-28fbf65aaf68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:23.412407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcabc3-90f9-43a6-b7e2-28fbf65aaf68 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:23.422845 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 766/1490] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:25.686627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4e4e3-337d-4cb3-8b1b-fc51166cc42b None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:25.687249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4e4e3-337d-4cb3-8b1b-fc51166cc42b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:25.687249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4e4e3-337d-4cb3-8b1b-fc51166cc42b None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:25.688639 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 725/1491] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 23 11:49:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:49:25.694427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00166a6c-8d35-4ae9-b19a-900c1e66a2c3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:25.695085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00166a6c-8d35-4ae9-b19a-900c1e66a2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:25.695085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00166a6c-8d35-4ae9-b19a-900c1e66a2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:25.713086 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-00166a6c-8d35-4ae9-b19a-900c1e66a2c3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:49:25.714350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-00166a6c-8d35-4ae9-b19a-900c1e66a2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:49:25.722557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 767/1492] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 23 11:49:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:49:25.762551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4e4e3-337d-4cb3-8b1b-fc51166cc42b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:49:25.773372 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4e4e3-337d-4cb3-8b1b-fc51166cc42b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:49:25.777078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc193002-4685-4c68-9cf7-edd3ad7a853d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:49:25.777732 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc193002-4685-4c68-9cf7-edd3ad7a853d service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:49:25.778020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc193002-4685-4c68-9cf7-edd3ad7a853d service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:49:25.778503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc193002-4685-4c68-9cf7-edd3ad7a853d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:49:25.788187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc193002-4685-4c68-9cf7-edd3ad7a853d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:49:25.789930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc193002-4685-4c68-9cf7-edd3ad7a853d service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:49:25.800461 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 726/1493] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:49:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:49:35.708355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=33 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:49:35.796004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=33 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:52.271556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47225898-dadb-4db1-a601-664723b2930f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.272179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47225898-dadb-4db1-a601-664723b2930f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.272179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47225898-dadb-4db1-a601-664723b2930f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.290296 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-47225898-dadb-4db1-a601-664723b2930f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:50:52.291488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-47225898-dadb-4db1-a601-664723b2930f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:52.299863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 768/1494] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:50:52.308960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a907b3-251e-4b71-9dc9-eb150d02e8a9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.309462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a907b3-251e-4b71-9dc9-eb150d02e8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.309576 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a907b3-251e-4b71-9dc9-eb150d02e8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.326869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-14a907b3-251e-4b71-9dc9-eb150d02e8a9 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:50:52.328063 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-14a907b3-251e-4b71-9dc9-eb150d02e8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:52.335803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 727/1495] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:50:52.340301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47225898-dadb-4db1-a601-664723b2930f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:52.350339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47225898-dadb-4db1-a601-664723b2930f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:52.355023 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783396bb-91ba-48ef-9220-3471af0b7b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:52.355682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783396bb-91ba-48ef-9220-3471af0b7b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.355970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783396bb-91ba-48ef-9220-3471af0b7b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.356251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783396bb-91ba-48ef-9220-3471af0b7b02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.357673 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783396bb-91ba-48ef-9220-3471af0b7b02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:50:52.357985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783396bb-91ba-48ef-9220-3471af0b7b02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:52.359024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-783396bb-91ba-48ef-9220-3471af0b7b02 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:50:52.359024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jun 23 11:50:52.359024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: Jun 23 11:50:52.359899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783396bb-91ba-48ef-9220-3471af0b7b02 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:52.371528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 769/1496] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:50:52.379625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a907b3-251e-4b71-9dc9-eb150d02e8a9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:52.389077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a907b3-251e-4b71-9dc9-eb150d02e8a9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:52.393683 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489dcf9-d45f-4bd4-b890-3dec275c37f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:52.394332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489dcf9-d45f-4bd4-b890-3dec275c37f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.394619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489dcf9-d45f-4bd4-b890-3dec275c37f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.394899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6489dcf9-d45f-4bd4-b890-3dec275c37f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.395622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489dcf9-d45f-4bd4-b890-3dec275c37f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:52.397454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-6489dcf9-d45f-4bd4-b890-3dec275c37f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:50:52.397454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jun 23 11:50:52.397454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: Jun 23 11:50:52.398655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6489dcf9-d45f-4bd4-b890-3dec275c37f2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:52.436980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 728/1497] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:50:52 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:50:52.441276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783396bb-91ba-48ef-9220-3471af0b7b02 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:52.451302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783396bb-91ba-48ef-9220-3471af0b7b02 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:52.505797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd90a04-ed21-49cd-b92e-5f583422b25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:52.506459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd90a04-ed21-49cd-b92e-5f583422b25b admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.506746 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd90a04-ed21-49cd-b92e-5f583422b25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.507027 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd90a04-ed21-49cd-b92e-5f583422b25b admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.507725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd90a04-ed21-49cd-b92e-5f583422b25b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:52.508755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-7fd90a04-ed21-49cd-b92e-5f583422b25b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:50:52.508755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jun 23 11:50:52.508755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: Jun 23 11:50:52.509952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd90a04-ed21-49cd-b92e-5f583422b25b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:52.530131 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 770/1498] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:50:52 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:50:52.534284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489dcf9-d45f-4bd4-b890-3dec275c37f2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:52.545946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6489dcf9-d45f-4bd4-b890-3dec275c37f2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:52.552041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b71f0-babf-4b8e-b090-86cca772fb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:52.552731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b71f0-babf-4b8e-b090-86cca772fb69 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.553018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b71f0-babf-4b8e-b090-86cca772fb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.553314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b71f0-babf-4b8e-b090-86cca772fb69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.554044 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b71f0-babf-4b8e-b090-86cca772fb69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:52.555035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-8c7b71f0-babf-4b8e-b090-86cca772fb69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:50:52.555035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jun 23 11:50:52.555035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: Jun 23 11:50:52.555909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b71f0-babf-4b8e-b090-86cca772fb69 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:52.568886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 729/1499] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:50:52.573298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd90a04-ed21-49cd-b92e-5f583422b25b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:52.583436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd90a04-ed21-49cd-b92e-5f583422b25b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:52.587884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45450c1f-8de8-4ef3-9093-79593b16316f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:52.588527 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45450c1f-8de8-4ef3-9093-79593b16316f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.588912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45450c1f-8de8-4ef3-9093-79593b16316f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.589192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45450c1f-8de8-4ef3-9093-79593b16316f admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.589911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45450c1f-8de8-4ef3-9093-79593b16316f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:52.590912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-45450c1f-8de8-4ef3-9093-79593b16316f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:50:52.590912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jun 23 11:50:52.590912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: Jun 23 11:50:52.592437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45450c1f-8de8-4ef3-9093-79593b16316f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:52.605613 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 771/1500] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:50:52.609738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b71f0-babf-4b8e-b090-86cca772fb69 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:52.619639 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b71f0-babf-4b8e-b090-86cca772fb69 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:52.624166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e7769b-1434-4409-b9e7-9c6b6efaea9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:52.625099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7769b-1434-4409-b9e7-9c6b6efaea9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.625383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7769b-1434-4409-b9e7-9c6b6efaea9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.625661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e7769b-1434-4409-b9e7-9c6b6efaea9e admin admin] PATH_INFO: `/v3/projects/b6137cff2e32425fa461f9d151e13990/users/7388ec7be25d42ec9ab097967a9ca347/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.640945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7769b-1434-4409-b9e7-9c6b6efaea9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6137cff2e32425fa461f9d151e13990, user_id=7388ec7be25d42ec9ab097967a9ca347, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:52.641977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-b8e7769b-1434-4409-b9e7-9c6b6efaea9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:50:52.641977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jun 23 11:50:52.641977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: Jun 23 11:50:52.643879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e7769b-1434-4409-b9e7-9c6b6efaea9e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:52.659354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 730/1501] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:50:52 2022] PUT /identity/v3/projects/b6137cff2e32425fa461f9d151e13990/users/7388ec7be25d42ec9ab097967a9ca347/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:50:52.672153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acd9b1-dd42-4807-a427-08ea8df2588a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.672448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acd9b1-dd42-4807-a427-08ea8df2588a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.672760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acd9b1-dd42-4807-a427-08ea8df2588a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.698062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-37acd9b1-dd42-4807-a427-08ea8df2588a None None] MFA Rules not processed for user `7388ec7be25d42ec9ab097967a9ca347`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:50:52.699278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-37acd9b1-dd42-4807-a427-08ea8df2588a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:52.762180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 772/1502] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:50:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:50:52.776100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339891d-5820-48cb-b353-8449bc1621e8 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.776395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339891d-5820-48cb-b353-8449bc1621e8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.776701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2339891d-5820-48cb-b353-8449bc1621e8 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.778081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 731/1503] 10.222.0.67 () {54 vars in 963 bytes} [Thu Jun 23 11:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:50:52.783906 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca69a-45bc-4e22-95c1-ef1e830c9b76 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.784190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca69a-45bc-4e22-95c1-ef1e830c9b76 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.784472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca69a-45bc-4e22-95c1-ef1e830c9b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.802850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f17ca69a-45bc-4e22-95c1-ef1e830c9b76 None None] MFA Rules not processed for user `abcbb404d9a148568a82697abd51ec30`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:50:52.804072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f17ca69a-45bc-4e22-95c1-ef1e830c9b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:52.847425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:52.855656 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 773/1504] 10.222.0.67 () {58 vars in 1060 bytes} [Thu Jun 23 11:50:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:50:52.863306 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339891d-5820-48cb-b353-8449bc1621e8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:52.874151 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2339891d-5820-48cb-b353-8449bc1621e8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:52.877710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed1c9f-ab9c-4cdb-bf21-4fed70ae9ddb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:52.878339 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed1c9f-ab9c-4cdb-bf21-4fed70ae9ddb service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.878627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed1c9f-ab9c-4cdb-bf21-4fed70ae9ddb service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.878907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed1c9f-ab9c-4cdb-bf21-4fed70ae9ddb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.887137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed1c9f-ab9c-4cdb-bf21-4fed70ae9ddb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:52.889087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed1c9f-ab9c-4cdb-bf21-4fed70ae9ddb service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:52.900000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 732/1505] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:50:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:50:52.945678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4c70a-8e74-4d13-b058-208dd480a108 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:52.945974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4c70a-8e74-4d13-b058-208dd480a108 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:52.946258 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4c70a-8e74-4d13-b058-208dd480a108 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:52.947554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 774/1506] 10.222.0.67 () {52 vars in 829 bytes} [Thu Jun 23 11:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:50:52.952525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed1c9f-ab9c-4cdb-bf21-4fed70ae9ddb service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:53.015129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed1c9f-ab9c-4cdb-bf21-4fed70ae9ddb service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:53.019712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:53.020570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:53.020889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:53.021165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] PATH_INFO: `/v3/users/7388ec7be25d42ec9ab097967a9ca347` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:53.022358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7388ec7be25d42ec9ab097967a9ca347)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:53.023377 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:50:53.023377 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jun 23 11:50:53.023377 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: Jun 23 11:50:53.024578 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:53.097317 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7388ec7be25d42ec9ab097967a9ca347'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:53.098044 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7388ec7be25d42ec9ab097967a9ca347'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:53.098666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7388ec7be25d42ec9ab097967a9ca347'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:53.117418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7388ec7be25d42ec9ab097967a9ca347'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:53.130012 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7388ec7be25d42ec9ab097967a9ca347'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:53.133995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:50:53.148650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 733/1507] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:50:52 2022] DELETE /identity/v3/users/7388ec7be25d42ec9ab097967a9ca347 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:50:53.152986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c55580-c53e-4a95-a880-9f66e7c7dc4f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:53.153284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c55580-c53e-4a95-a880-9f66e7c7dc4f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:53.153592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c55580-c53e-4a95-a880-9f66e7c7dc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:53.172016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-65c55580-c53e-4a95-a880-9f66e7c7dc4f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:50:53.173257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-65c55580-c53e-4a95-a880-9f66e7c7dc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:53.238207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 775/1508] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:50:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:50:53.249196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:53.250650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:53.306881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886dbe7-02bb-4d1a-8a8a-c9077c8a09c4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:53.311028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf79fc-62e5-46ff-a44a-2c44037eac81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:53.311664 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf79fc-62e5-46ff-a44a-2c44037eac81 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:53.311953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf79fc-62e5-46ff-a44a-2c44037eac81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:53.312224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf79fc-62e5-46ff-a44a-2c44037eac81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:53.320936 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf79fc-62e5-46ff-a44a-2c44037eac81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:53.322355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf79fc-62e5-46ff-a44a-2c44037eac81 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:53.331514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 734/1509] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:50:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:50:53.666195 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c55580-c53e-4a95-a880-9f66e7c7dc4f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:50:53.667654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-65c55580-c53e-4a95-a880-9f66e7c7dc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:50:53.681430 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c55580-c53e-4a95-a880-9f66e7c7dc4f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:50:53.686324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:50:53.687181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:50:53.687459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:50:53.687730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] PATH_INFO: `/v3/projects/b6137cff2e32425fa461f9d151e13990` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:50:53.688956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6137cff2e32425fa461f9d151e13990)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:50:53.689973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:50:53.689973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jun 23 11:50:53.689973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: Jun 23 11:50:53.691146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:50:53.752982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6137cff2e32425fa461f9d151e13990', 'request_id': 'req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:53.762528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6137cff2e32425fa461f9d151e13990', 'request_id': 'req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:53.769161 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6137cff2e32425fa461f9d151e13990', 'request_id': 'req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:53.787473 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] The token cache is being invalidate because project b6137cff2e32425fa461f9d151e13990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:50:53.787763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:53.788505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7ec00250-4c8b-4fe5-aadc-1b28d44aaef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:50:53.790344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 776/1510] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:50:53 2022] DELETE /identity/v3/projects/b6137cff2e32425fa461f9d151e13990 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:51:03.327487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:51:03.784491 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:33.921028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32389828-c7d6-4814-9ef1-04cb23774632 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:33.921600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32389828-c7d6-4814-9ef1-04cb23774632 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:33.921600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32389828-c7d6-4814-9ef1-04cb23774632 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:33.923027 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 735/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:52:33.928370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d82b47-cec4-480a-851d-5e5e22888eed None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:33.928882 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d82b47-cec4-480a-851d-5e5e22888eed None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:33.929005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d82b47-cec4-480a-851d-5e5e22888eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:33.946507 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a6d82b47-cec4-480a-851d-5e5e22888eed None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:33.947722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a6d82b47-cec4-480a-851d-5e5e22888eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:34.006301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 777/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:34.160196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32389828-c7d6-4814-9ef1-04cb23774632 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:34.161997 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-32389828-c7d6-4814-9ef1-04cb23774632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:34.235940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32389828-c7d6-4814-9ef1-04cb23774632 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:34.241453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441c0c0-b0da-4b63-91df-34f71b865827 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:34.242105 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441c0c0-b0da-4b63-91df-34f71b865827 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:34.242394 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441c0c0-b0da-4b63-91df-34f71b865827 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:34.242675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441c0c0-b0da-4b63-91df-34f71b865827 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:34.252055 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441c0c0-b0da-4b63-91df-34f71b865827 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:34.253657 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441c0c0-b0da-4b63-91df-34f71b865827 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:34.263071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 736/1513] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:35.109297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fb238-ed61-4793-b275-ead32d384048 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:35.109864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fb238-ed61-4793-b275-ead32d384048 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:35.109864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fb238-ed61-4793-b275-ead32d384048 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:35.111257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 778/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:52:35.116048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0362ea-e8b5-4795-acd1-fc5fa13e85f0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:35.116456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0362ea-e8b5-4795-acd1-fc5fa13e85f0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:35.116621 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0362ea-e8b5-4795-acd1-fc5fa13e85f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:35.135276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ad0362ea-e8b5-4795-acd1-fc5fa13e85f0 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:35.136502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ad0362ea-e8b5-4795-acd1-fc5fa13e85f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:35.145039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 737/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:35.302502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fb238-ed61-4793-b275-ead32d384048 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:35.317464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fb238-ed61-4793-b275-ead32d384048 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:35.323093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b4f0c-b113-44a7-b742-5c236d9a3f8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:35.323734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b4f0c-b113-44a7-b742-5c236d9a3f8c service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:35.324023 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b4f0c-b113-44a7-b742-5c236d9a3f8c service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:35.324302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b4f0c-b113-44a7-b742-5c236d9a3f8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:35.332597 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b4f0c-b113-44a7-b742-5c236d9a3f8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:35.334256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b4f0c-b113-44a7-b742-5c236d9a3f8c service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:35.345159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 779/1516] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:52:35.908943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8824d-ffdf-4cc8-b3e0-4b1530a4c5b4 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:35.909241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8824d-ffdf-4cc8-b3e0-4b1530a4c5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:35.909529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8824d-ffdf-4cc8-b3e0-4b1530a4c5b4 None None] PATH_INFO: `/v3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:35.911004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 738/1517] 10.222.0.44 () {54 vars in 925 bytes} [Thu Jun 23 11:52:35 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:52:35.917479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcdd94-910e-4d1a-aca2-a78efb84068d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:35.917827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcdd94-910e-4d1a-aca2-a78efb84068d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:35.918062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcdd94-910e-4d1a-aca2-a78efb84068d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:35.936055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5ffcdd94-910e-4d1a-aca2-a78efb84068d None None] MFA Rules not processed for user `ee971347d23040d3984f961025e8157e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:35.937307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5ffcdd94-910e-4d1a-aca2-a78efb84068d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:36.000103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 780/1518] 10.222.0.44 () {58 vars in 1015 bytes} [Thu Jun 23 11:52:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:39.801310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2b42f-4d37-449e-ad58-e867c7f5617f None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.801833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2b42f-4d37-449e-ad58-e867c7f5617f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.801833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2b42f-4d37-449e-ad58-e867c7f5617f None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.803198 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 739/1519] 10.222.0.67 () {54 vars in 960 bytes} [Thu Jun 23 11:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:52:39.808328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990bd4c-486f-49d2-a387-64f153ad92fa None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.808794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990bd4c-486f-49d2-a387-64f153ad92fa None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.808906 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b990bd4c-486f-49d2-a387-64f153ad92fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.826725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b990bd4c-486f-49d2-a387-64f153ad92fa None None] MFA Rules not processed for user `ee971347d23040d3984f961025e8157e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:39.827955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b990bd4c-486f-49d2-a387-64f153ad92fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:39.836864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 781/1520] 10.222.0.67 () {58 vars in 1057 bytes} [Thu Jun 23 11:52:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:39.844247 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2b42f-4d37-449e-ad58-e867c7f5617f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:39.859004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2b42f-4d37-449e-ad58-e867c7f5617f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:39.865167 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5e015-d8dd-4c4f-96ab-268f31c7ae8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:39.866450 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5e015-d8dd-4c4f-96ab-268f31c7ae8f service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:39.866979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5e015-d8dd-4c4f-96ab-268f31c7ae8f service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:39.867488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5e015-d8dd-4c4f-96ab-268f31c7ae8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:39.878316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5e015-d8dd-4c4f-96ab-268f31c7ae8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:39.879904 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae5e015-d8dd-4c4f-96ab-268f31c7ae8f service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:39.890298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 740/1521] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:46.589606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff586d74-3a57-4b01-a093-c89a1876ee29 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:46.590336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff586d74-3a57-4b01-a093-c89a1876ee29 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:46.590336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff586d74-3a57-4b01-a093-c89a1876ee29 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:46.591570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 782/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:52:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:52:46.596644 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d488d7-85e5-4013-a71a-9e6fb399ecb6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:46.597142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d488d7-85e5-4013-a71a-9e6fb399ecb6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:46.597295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d488d7-85e5-4013-a71a-9e6fb399ecb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:46.615579 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b0d488d7-85e5-4013-a71a-9e6fb399ecb6 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:46.616841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b0d488d7-85e5-4013-a71a-9e6fb399ecb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:46.624768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 741/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:52:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:52:46.778842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff586d74-3a57-4b01-a093-c89a1876ee29 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:46.794085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff586d74-3a57-4b01-a093-c89a1876ee29 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:46.799524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e204c2-3c3e-4536-a24f-325983fd6a10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:46.800159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e204c2-3c3e-4536-a24f-325983fd6a10 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:46.800448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e204c2-3c3e-4536-a24f-325983fd6a10 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:46.800764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e204c2-3c3e-4536-a24f-325983fd6a10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:46.809503 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e204c2-3c3e-4536-a24f-325983fd6a10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:46.811070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e204c2-3c3e-4536-a24f-325983fd6a10 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:46.820376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 783/1524] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:47.645282 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500dfa8-7c1e-40fe-b4b9-dcb428a27259 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:47.645946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500dfa8-7c1e-40fe-b4b9-dcb428a27259 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:47.645946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500dfa8-7c1e-40fe-b4b9-dcb428a27259 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:47.647821 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 742/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 11:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:52:47.653510 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f5ff3-708a-4a82-9963-e4b94511acf2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:47.653983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f5ff3-708a-4a82-9963-e4b94511acf2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:47.654087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35f5ff3-708a-4a82-9963-e4b94511acf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:47.672660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c35f5ff3-708a-4a82-9963-e4b94511acf2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:47.673905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c35f5ff3-708a-4a82-9963-e4b94511acf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:47.682093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 784/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 11:52:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:47.834092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500dfa8-7c1e-40fe-b4b9-dcb428a27259 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:47.848562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500dfa8-7c1e-40fe-b4b9-dcb428a27259 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:47.853966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2700f18-93a8-451e-9567-d6f6efd5d08b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:47.854631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2700f18-93a8-451e-9567-d6f6efd5d08b service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:47.854912 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2700f18-93a8-451e-9567-d6f6efd5d08b service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:47.855214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2700f18-93a8-451e-9567-d6f6efd5d08b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:47.864294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2700f18-93a8-451e-9567-d6f6efd5d08b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:47.866005 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2700f18-93a8-451e-9567-d6f6efd5d08b service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:47.875785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 743/1527] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:51.790872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16650e4-4ab9-4754-9555-be192f908ac6 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:51.791732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16650e4-4ab9-4754-9555-be192f908ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:51.791732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16650e4-4ab9-4754-9555-be192f908ac6 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:51.793178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 785/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:52:51.798904 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176f44-a1b3-4415-8ffe-d1400ec6e74b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:51.799422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176f44-a1b3-4415-8ffe-d1400ec6e74b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:51.799566 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176f44-a1b3-4415-8ffe-d1400ec6e74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:51.820798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3e176f44-a1b3-4415-8ffe-d1400ec6e74b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:51.822025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3e176f44-a1b3-4415-8ffe-d1400ec6e74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:51.829926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 744/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:51.866534 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16650e4-4ab9-4754-9555-be192f908ac6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:51.868399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c16650e4-4ab9-4754-9555-be192f908ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:51.933333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16650e4-4ab9-4754-9555-be192f908ac6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:51.937186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a85ce-c0bf-4796-b2cd-e0282157d263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:51.937909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a85ce-c0bf-4796-b2cd-e0282157d263 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:51.938248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a85ce-c0bf-4796-b2cd-e0282157d263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:51.938531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a85ce-c0bf-4796-b2cd-e0282157d263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:51.948869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a85ce-c0bf-4796-b2cd-e0282157d263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:51.950311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a85ce-c0bf-4796-b2cd-e0282157d263 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:51.955319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:52:51.960344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 786/1530] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:54.006366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a7719-346d-4f07-a766-f1c996eea8f3 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:54.006955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a7719-346d-4f07-a766-f1c996eea8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:54.006955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a7719-346d-4f07-a766-f1c996eea8f3 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:54.008349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 745/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:52:54.014126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66cdfa-f058-42bd-8e49-def60741a715 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:54.014573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66cdfa-f058-42bd-8e49-def60741a715 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:54.014675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66cdfa-f058-42bd-8e49-def60741a715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:54.033596 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3c66cdfa-f058-42bd-8e49-def60741a715 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:54.034818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3c66cdfa-f058-42bd-8e49-def60741a715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:54.043708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 787/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:54.083690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a7719-346d-4f07-a766-f1c996eea8f3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:54.085652 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-786a7719-346d-4f07-a766-f1c996eea8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:54.099906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a7719-346d-4f07-a766-f1c996eea8f3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:54.103696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d3e90-f24e-41ab-b0c3-f9fa8a70ce83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:54.104334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3e90-f24e-41ab-b0c3-f9fa8a70ce83 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:54.104938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3e90-f24e-41ab-b0c3-f9fa8a70ce83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:54.105223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3e90-f24e-41ab-b0c3-f9fa8a70ce83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:54.114569 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d3e90-f24e-41ab-b0c3-f9fa8a70ce83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:54.116004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d3e90-f24e-41ab-b0c3-f9fa8a70ce83 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:54.125936 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 746/1533] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:52:57.804801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc21448-007c-4de1-b2c9-6a8dbc723e90 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:57.805363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc21448-007c-4de1-b2c9-6a8dbc723e90 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:57.805363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc21448-007c-4de1-b2c9-6a8dbc723e90 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:57.806714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 788/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:52:57.812174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299034c-9eec-4e76-b029-a3cec87965e3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:57.812579 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299034c-9eec-4e76-b029-a3cec87965e3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:57.812763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299034c-9eec-4e76-b029-a3cec87965e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:57.831730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8299034c-9eec-4e76-b029-a3cec87965e3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:52:57.833716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8299034c-9eec-4e76-b029-a3cec87965e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:57.842152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 747/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:52:58.146550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc21448-007c-4de1-b2c9-6a8dbc723e90 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:52:58.148226 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3fc21448-007c-4de1-b2c9-6a8dbc723e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:52:58.204370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc21448-007c-4de1-b2c9-6a8dbc723e90 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:52:58.208494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e0cd4d-8787-4f5a-9b2d-1075210b8afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:52:58.209158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0cd4d-8787-4f5a-9b2d-1075210b8afd service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:52:58.209444 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0cd4d-8787-4f5a-9b2d-1075210b8afd service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:52:58.209721 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0cd4d-8787-4f5a-9b2d-1075210b8afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:52:58.218523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0cd4d-8787-4f5a-9b2d-1075210b8afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:52:58.219930 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e0cd4d-8787-4f5a-9b2d-1075210b8afd service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:52:58.229284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 789/1536] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:53:00.283802 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8f60d-d497-48e7-8e3a-88080a9f9611 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:00.284318 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8f60d-d497-48e7-8e3a-88080a9f9611 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:00.284428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8f60d-d497-48e7-8e3a-88080a9f9611 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:00.285806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 748/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 11:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:53:00.291097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a20e2d-5317-4c77-b826-b3c61947e6a8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:00.291796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a20e2d-5317-4c77-b826-b3c61947e6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:00.291919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a20e2d-5317-4c77-b826-b3c61947e6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:00.310375 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-89a20e2d-5317-4c77-b826-b3c61947e6a8 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:53:00.311598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-89a20e2d-5317-4c77-b826-b3c61947e6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:00.319980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 790/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 11:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:53:00.627106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8f60d-d497-48e7-8e3a-88080a9f9611 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:53:00.629149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9cf8f60d-d497-48e7-8e3a-88080a9f9611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:53:00.642799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8f60d-d497-48e7-8e3a-88080a9f9611 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:53:00.646369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabdeb30-3718-4960-ad5a-af15f4f7796c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:53:00.647056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdeb30-3718-4960-ad5a-af15f4f7796c service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:53:00.647283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdeb30-3718-4960-ad5a-af15f4f7796c service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:53:00.647560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabdeb30-3718-4960-ad5a-af15f4f7796c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:53:00.656488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdeb30-3718-4960-ad5a-af15f4f7796c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:53:00.657948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabdeb30-3718-4960-ad5a-af15f4f7796c service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:53:00.668099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 749/1539] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:53:10.305186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=32 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:53:10.663401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=70 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:33.187560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b6519-da8b-4c1e-82de-256abb3a8294 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.188251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b6519-da8b-4c1e-82de-256abb3a8294 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.188251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b6519-da8b-4c1e-82de-256abb3a8294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.212694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-280b6519-da8b-4c1e-82de-256abb3a8294 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:33.213868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-280b6519-da8b-4c1e-82de-256abb3a8294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:33.225051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 791/1540] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:54:33.235480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e443d-cdd0-478b-b511-08f5a8eb34d0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.236526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e443d-cdd0-478b-b511-08f5a8eb34d0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.237061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e443d-cdd0-478b-b511-08f5a8eb34d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.265064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b82e443d-cdd0-478b-b511-08f5a8eb34d0 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:33.266831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b82e443d-cdd0-478b-b511-08f5a8eb34d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:33.276266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 750/1541] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:33.281171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b6519-da8b-4c1e-82de-256abb3a8294 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:33.296167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b6519-da8b-4c1e-82de-256abb3a8294 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:33.300923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e2130-e975-47e3-88d2-841fda17b565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:33.301600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e2130-e975-47e3-88d2-841fda17b565 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.301882 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e2130-e975-47e3-88d2-841fda17b565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.302170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e2130-e975-47e3-88d2-841fda17b565 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.309066 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e2130-e975-47e3-88d2-841fda17b565 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:54:33.309447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e2130-e975-47e3-88d2-841fda17b565 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:33.311109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e2130-e975-47e3-88d2-841fda17b565 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:33.324146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 792/1542] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:54:33.332747 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e443d-cdd0-478b-b511-08f5a8eb34d0 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:33.343787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e443d-cdd0-478b-b511-08f5a8eb34d0 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:33.355782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b33e9f-4c10-4aaa-833c-361b53b4a5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:33.356886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b33e9f-4c10-4aaa-833c-361b53b4a5da admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.357415 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b33e9f-4c10-4aaa-833c-361b53b4a5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.357921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b33e9f-4c10-4aaa-833c-361b53b4a5da admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.359494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b33e9f-4c10-4aaa-833c-361b53b4a5da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:33.361752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b33e9f-4c10-4aaa-833c-361b53b4a5da admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:33.419711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 751/1543] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:33 2022] POST /identity/v3/projects => generated 403 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:54:33.424245 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e2130-e975-47e3-88d2-841fda17b565 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:33.447055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e2130-e975-47e3-88d2-841fda17b565 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:33.525117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e964881-22e7-4793-8cc5-78451340b225 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.525815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e964881-22e7-4793-8cc5-78451340b225 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.526334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e964881-22e7-4793-8cc5-78451340b225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.540027 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43dc32-dc18-455f-bdfe-e4726b4691dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:33.540736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43dc32-dc18-455f-bdfe-e4726b4691dc admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.541033 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43dc32-dc18-455f-bdfe-e4726b4691dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.541319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43dc32-dc18-455f-bdfe-e4726b4691dc admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.542034 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43dc32-dc18-455f-bdfe-e4726b4691dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:33.544386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43dc32-dc18-455f-bdfe-e4726b4691dc admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:33.559591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-5e964881-22e7-4793-8cc5-78451340b225 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:33.562971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5e964881-22e7-4793-8cc5-78451340b225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:33.571617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 793/1544] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:33 2022] POST /identity/v3/users => generated 302 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:33.575626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43dc32-dc18-455f-bdfe-e4726b4691dc admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:33.581702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 752/1545] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:54:33.587681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43dc32-dc18-455f-bdfe-e4726b4691dc admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:33.592847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b928554-501e-469a-bab4-05821f3a9a01 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.593419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b928554-501e-469a-bab4-05821f3a9a01 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.594342 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b928554-501e-469a-bab4-05821f3a9a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.597897 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348fbe0-3fb1-430b-bc4d-badbe032e222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:33.598580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348fbe0-3fb1-430b-bc4d-badbe032e222 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.598878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348fbe0-3fb1-430b-bc4d-badbe032e222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.599160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e348fbe0-3fb1-430b-bc4d-badbe032e222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.599902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348fbe0-3fb1-430b-bc4d-badbe032e222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:33.602130 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e348fbe0-3fb1-430b-bc4d-badbe032e222 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:33.618211 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 794/1546] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:33.622155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348fbe0-3fb1-430b-bc4d-badbe032e222 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:33.625650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9b928554-501e-469a-bab4-05821f3a9a01 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:33.627408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9b928554-501e-469a-bab4-05821f3a9a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:33.634806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e348fbe0-3fb1-430b-bc4d-badbe032e222 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:33.649902 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 753/1547] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:54:33.653959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4282f900-1b2e-4338-8efe-d9a222e09141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:33.654541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b928554-501e-469a-bab4-05821f3a9a01 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:33.654701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4282f900-1b2e-4338-8efe-d9a222e09141 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.654929 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4282f900-1b2e-4338-8efe-d9a222e09141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.655210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4282f900-1b2e-4338-8efe-d9a222e09141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.655974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4282f900-1b2e-4338-8efe-d9a222e09141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:33.657682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4282f900-1b2e-4338-8efe-d9a222e09141 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:33.677881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b928554-501e-469a-bab4-05821f3a9a01 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:33.685614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 795/1548] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:33 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:54:33.689611 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4282f900-1b2e-4338-8efe-d9a222e09141 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:33.698682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7ef7f-bcad-44b5-a577-579438526feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:33.699422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7ef7f-bcad-44b5-a577-579438526feb admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.699716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7ef7f-bcad-44b5-a577-579438526feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.700004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7ef7f-bcad-44b5-a577-579438526feb admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.701961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7ef7f-bcad-44b5-a577-579438526feb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:54:33.702284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7ef7f-bcad-44b5-a577-579438526feb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:33.703387 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-f8c7ef7f-bcad-44b5-a577-579438526feb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:54:33.703387 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jun 23 11:54:33.703387 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: Jun 23 11:54:33.704342 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7ef7f-bcad-44b5-a577-579438526feb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:33.716150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4282f900-1b2e-4338-8efe-d9a222e09141 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:33.732425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 754/1549] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:54:33.739562 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6ade82-39a3-4d71-9049-46a210c54751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:33.740057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7ef7f-bcad-44b5-a577-579438526feb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:33.740488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ade82-39a3-4d71-9049-46a210c54751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.740794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ade82-39a3-4d71-9049-46a210c54751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.741083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6ade82-39a3-4d71-9049-46a210c54751 admin admin] PATH_INFO: `/v3/projects/be6f9428571347cdbe36e96b7f441522/users/7b9a57f7f87748fc916386de21821c8b/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.763421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7ef7f-bcad-44b5-a577-579438526feb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:33.767803 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ade82-39a3-4d71-9049-46a210c54751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be6f9428571347cdbe36e96b7f441522, user_id=7b9a57f7f87748fc916386de21821c8b, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:33.768908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-7e6ade82-39a3-4d71-9049-46a210c54751 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:54:33.768908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jun 23 11:54:33.768908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: Jun 23 11:54:33.771226 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6ade82-39a3-4d71-9049-46a210c54751 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:33.786250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27796a3-048d-4226-b6fd-1e248a6d7090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:33.786953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27796a3-048d-4226-b6fd-1e248a6d7090 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.787157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27796a3-048d-4226-b6fd-1e248a6d7090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.787508 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27796a3-048d-4226-b6fd-1e248a6d7090 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.788601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27796a3-048d-4226-b6fd-1e248a6d7090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:33.791746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27796a3-048d-4226-b6fd-1e248a6d7090 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:33.801095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 796/1550] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:33 2022] PUT /identity/v3/projects/be6f9428571347cdbe36e96b7f441522/users/7b9a57f7f87748fc916386de21821c8b/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:54:33.802869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e57e1-2130-40fa-90b2-9bc31b249e65 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.803167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e57e1-2130-40fa-90b2-9bc31b249e65 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.803454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e57e1-2130-40fa-90b2-9bc31b249e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.843360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-172e57e1-2130-40fa-90b2-9bc31b249e65 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:33.844638 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-172e57e1-2130-40fa-90b2-9bc31b249e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:33.854265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 755/1551] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:33 2022] POST /identity/v3/projects => generated 393 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:33.855754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c836ff3-3e23-4603-9a57-8a68f74f220d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:33.856054 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c836ff3-3e23-4603-9a57-8a68f74f220d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:33.856336 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c836ff3-3e23-4603-9a57-8a68f74f220d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:33.891150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1c836ff3-3e23-4603-9a57-8a68f74f220d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:33.893022 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1c836ff3-3e23-4603-9a57-8a68f74f220d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:33.967111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 797/1552] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:33.967933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e57e1-2130-40fa-90b2-9bc31b249e65 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:33.987631 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e57e1-2130-40fa-90b2-9bc31b249e65 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.014389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 756/1553] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:34.016047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a0881-4080-4c72-88aa-ef883610d54d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.016346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a0881-4080-4c72-88aa-ef883610d54d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.016633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a0881-4080-4c72-88aa-ef883610d54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.031039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:34.048156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a46a0881-4080-4c72-88aa-ef883610d54d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:34.049694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a46a0881-4080-4c72-88aa-ef883610d54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:34.066593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 757/1554] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:34.068208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cc183-6450-48b7-9f6e-38a3a4e43048 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.068511 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cc183-6450-48b7-9f6e-38a3a4e43048 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.068925 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cc183-6450-48b7-9f6e-38a3a4e43048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.093420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f73cc183-6450-48b7-9f6e-38a3a4e43048 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:34.094003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44cf38-a72e-41e8-9c50-b6d4f08da053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.094994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44cf38-a72e-41e8-9c50-b6d4f08da053 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.095285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44cf38-a72e-41e8-9c50-b6d4f08da053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.095723 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44cf38-a72e-41e8-9c50-b6d4f08da053 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.095959 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f73cc183-6450-48b7-9f6e-38a3a4e43048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:34.096462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44cf38-a72e-41e8-9c50-b6d4f08da053 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.098799 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44cf38-a72e-41e8-9c50-b6d4f08da053 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.106513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 758/1555] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:34.108312 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f5531-b972-4f3a-8b58-ab253af89c29 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.108847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f5531-b972-4f3a-8b58-ab253af89c29 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.109336 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f5531-b972-4f3a-8b58-ab253af89c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.133787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 798/1556] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:33 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:34.137586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-828f5531-b972-4f3a-8b58-ab253af89c29 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:34.137586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-828f5531-b972-4f3a-8b58-ab253af89c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:34.138803 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44cf38-a72e-41e8-9c50-b6d4f08da053 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.148737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 759/1557] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:34.148737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0406fa-7306-4675-bdb3-60fbfa32f390 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.148737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0406fa-7306-4675-bdb3-60fbfa32f390 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.148737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0406fa-7306-4675-bdb3-60fbfa32f390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.170385 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ba0406fa-7306-4675-bdb3-60fbfa32f390 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:34.171716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ba0406fa-7306-4675-bdb3-60fbfa32f390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:34.188725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 760/1558] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:34.188725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0406fa-7306-4675-bdb3-60fbfa32f390 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.198393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0406fa-7306-4675-bdb3-60fbfa32f390 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.203877 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a56363-33fd-4695-9999-f2d705cf9d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.204547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a56363-33fd-4695-9999-f2d705cf9d0f admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.205853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a56363-33fd-4695-9999-f2d705cf9d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.205853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a56363-33fd-4695-9999-f2d705cf9d0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.206103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a56363-33fd-4695-9999-f2d705cf9d0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:54:34.206324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a56363-33fd-4695-9999-f2d705cf9d0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.207973 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a56363-33fd-4695-9999-f2d705cf9d0f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.223060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44cf38-a72e-41e8-9c50-b6d4f08da053 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.224851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 761/1559] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:54:34.226311 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3cfda-38c5-45fe-84b9-2fff7fd784b2 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.227115 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3cfda-38c5-45fe-84b9-2fff7fd784b2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.227396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3cfda-38c5-45fe-84b9-2fff7fd784b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.228897 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30c7cd-7452-4a02-be31-91ba6d3d3964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.229751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30c7cd-7452-4a02-be31-91ba6d3d3964 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.230035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30c7cd-7452-4a02-be31-91ba6d3d3964 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.230318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30c7cd-7452-4a02-be31-91ba6d3d3964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.240550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30c7cd-7452-4a02-be31-91ba6d3d3964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.242089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30c7cd-7452-4a02-be31-91ba6d3d3964 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.246589 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9dd3cfda-38c5-45fe-84b9-2fff7fd784b2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:34.247850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9dd3cfda-38c5-45fe-84b9-2fff7fd784b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:34.252741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 799/1560] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:34.253577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30c7cd-7452-4a02-be31-91ba6d3d3964 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.260042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 762/1561] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:34.260042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc53300-8f15-4dd5-95fb-4aa0471e5a56 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.260042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc53300-8f15-4dd5-95fb-4aa0471e5a56 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.260042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc53300-8f15-4dd5-95fb-4aa0471e5a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.263296 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30c7cd-7452-4a02-be31-91ba6d3d3964 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.268376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c417e27-a3f7-476c-b931-fe995e3ce9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.269072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c417e27-a3f7-476c-b931-fe995e3ce9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.269363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c417e27-a3f7-476c-b931-fe995e3ce9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.269640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c417e27-a3f7-476c-b931-fe995e3ce9e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.270511 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c417e27-a3f7-476c-b931-fe995e3ce9e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.271956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c417e27-a3f7-476c-b931-fe995e3ce9e6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.278919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-adc53300-8f15-4dd5-95fb-4aa0471e5a56 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:34.280321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-adc53300-8f15-4dd5-95fb-4aa0471e5a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:34.283387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 800/1562] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:34.284741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89729121-df71-42fb-9c9e-2ee1f794b288 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.285037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89729121-df71-42fb-9c9e-2ee1f794b288 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.285318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89729121-df71-42fb-9c9e-2ee1f794b288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.297919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 763/1563] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:34.299676 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee69ce-f325-4974-a1e7-a2d20cc15e04 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.299979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee69ce-f325-4974-a1e7-a2d20cc15e04 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.300192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee69ce-f325-4974-a1e7-a2d20cc15e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.305190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-89729121-df71-42fb-9c9e-2ee1f794b288 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:34.306409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-89729121-df71-42fb-9c9e-2ee1f794b288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:34.317223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 801/1564] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:34.318573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabde49-01f3-4f81-b2aa-43248e89f52a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.318874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabde49-01f3-4f81-b2aa-43248e89f52a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.319154 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabde49-01f3-4f81-b2aa-43248e89f52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.319804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b2ee69ce-f325-4974-a1e7-a2d20cc15e04 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:34.321661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b2ee69ce-f325-4974-a1e7-a2d20cc15e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:34.333234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 764/1565] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:34.334056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee69ce-f325-4974-a1e7-a2d20cc15e04 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.347030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5fabde49-01f3-4f81-b2aa-43248e89f52a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:34.348514 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5fabde49-01f3-4f81-b2aa-43248e89f52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:34.348738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee69ce-f325-4974-a1e7-a2d20cc15e04 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.354337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cb4e7-bb0f-49cd-afed-d188324c778b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.355433 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cb4e7-bb0f-49cd-afed-d188324c778b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.355813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cb4e7-bb0f-49cd-afed-d188324c778b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.355942 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cb4e7-bb0f-49cd-afed-d188324c778b admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.356640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cb4e7-bb0f-49cd-afed-d188324c778b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.358692 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 802/1566] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:34.358832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cb4e7-bb0f-49cd-afed-d188324c778b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.359534 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabde49-01f3-4f81-b2aa-43248e89f52a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.374316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabde49-01f3-4f81-b2aa-43248e89f52a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.379556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29a047-0a39-4bf2-bfb5-c9695d6ad477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.380386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29a047-0a39-4bf2-bfb5-c9695d6ad477 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.380683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29a047-0a39-4bf2-bfb5-c9695d6ad477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.380983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29a047-0a39-4bf2-bfb5-c9695d6ad477 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.381737 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29a047-0a39-4bf2-bfb5-c9695d6ad477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:54:34.382037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29a047-0a39-4bf2-bfb5-c9695d6ad477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.384194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad29a047-0a39-4bf2-bfb5-c9695d6ad477 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.392215 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 765/1567] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/projects => generated 389 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:34.393192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cb4e7-bb0f-49cd-afed-d188324c778b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.397129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 803/1568] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:54:34.399086 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29a047-0a39-4bf2-bfb5-c9695d6ad477 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.399178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:34.410405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29a047-0a39-4bf2-bfb5-c9695d6ad477 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.487608 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0125c-e7e0-4f2e-a21a-87b0ce04f846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.488695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0125c-e7e0-4f2e-a21a-87b0ce04f846 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.489202 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0125c-e7e0-4f2e-a21a-87b0ce04f846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.489698 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0125c-e7e0-4f2e-a21a-87b0ce04f846 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.490516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0125c-e7e0-4f2e-a21a-87b0ce04f846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:54:34.490899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0125c-e7e0-4f2e-a21a-87b0ce04f846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.493206 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0125c-e7e0-4f2e-a21a-87b0ce04f846 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.496420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cb4e7-bb0f-49cd-afed-d188324c778b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.501345 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf56fb8-9c1d-40fa-a504-c3d1a6b2017f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.502039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf56fb8-9c1d-40fa-a504-c3d1a6b2017f admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.502326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf56fb8-9c1d-40fa-a504-c3d1a6b2017f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.502609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf56fb8-9c1d-40fa-a504-c3d1a6b2017f admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.503351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf56fb8-9c1d-40fa-a504-c3d1a6b2017f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.505035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf56fb8-9c1d-40fa-a504-c3d1a6b2017f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.508615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 804/1569] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:54:34.510414 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1093d5-d5c1-4770-9b1b-e1ed357cb0d8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.510908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1093d5-d5c1-4770-9b1b-e1ed357cb0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.511370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1093d5-d5c1-4770-9b1b-e1ed357cb0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.525331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 766/1570] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:34.526313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf56fb8-9c1d-40fa-a504-c3d1a6b2017f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.535794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bd1093d5-d5c1-4770-9b1b-e1ed357cb0d8 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:34.537352 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf56fb8-9c1d-40fa-a504-c3d1a6b2017f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.538360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bd1093d5-d5c1-4770-9b1b-e1ed357cb0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:34.542264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74008dd1-d6d8-4eaf-8c72-1537511669a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.543193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74008dd1-d6d8-4eaf-8c72-1537511669a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.543479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74008dd1-d6d8-4eaf-8c72-1537511669a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.543756 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74008dd1-d6d8-4eaf-8c72-1537511669a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.544506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74008dd1-d6d8-4eaf-8c72-1537511669a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:54:34.545090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74008dd1-d6d8-4eaf-8c72-1537511669a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.546797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74008dd1-d6d8-4eaf-8c72-1537511669a8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.551262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 805/1571] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:34.553268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc05ec1-1e14-4fee-9110-44f6ac2b3a6e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.553739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc05ec1-1e14-4fee-9110-44f6ac2b3a6e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.555298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc05ec1-1e14-4fee-9110-44f6ac2b3a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.561731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 767/1572] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:54:34.562999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74008dd1-d6d8-4eaf-8c72-1537511669a8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.575989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74008dd1-d6d8-4eaf-8c72-1537511669a8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.576715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:34.582807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b81f7-9e02-4171-9db5-743987d0d569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.583629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b81f7-9e02-4171-9db5-743987d0d569 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.583980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b81f7-9e02-4171-9db5-743987d0d569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.584359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b81f7-9e02-4171-9db5-743987d0d569 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.584546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8cc05ec1-1e14-4fee-9110-44f6ac2b3a6e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:34.585238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b81f7-9e02-4171-9db5-743987d0d569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.586035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8cc05ec1-1e14-4fee-9110-44f6ac2b3a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:34.586419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-e43b81f7-9e02-4171-9db5-743987d0d569 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:54:34.586419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jun 23 11:54:34.586419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: Jun 23 11:54:34.587813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b81f7-9e02-4171-9db5-743987d0d569 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.595818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 806/1573] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:34.602736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc05ec1-1e14-4fee-9110-44f6ac2b3a6e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.615427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc05ec1-1e14-4fee-9110-44f6ac2b3a6e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.619741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 768/1574] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/users => generated 295 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:34.620841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b81f7-9e02-4171-9db5-743987d0d569 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.622763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65b414-5239-4c72-a366-ce6a95578112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.623702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65b414-5239-4c72-a366-ce6a95578112 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.624047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65b414-5239-4c72-a366-ce6a95578112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.624490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65b414-5239-4c72-a366-ce6a95578112 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.625419 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65b414-5239-4c72-a366-ce6a95578112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.626960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-9a65b414-5239-4c72-a366-ce6a95578112 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:54:34.626960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jun 23 11:54:34.626960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: Jun 23 11:54:34.628111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a65b414-5239-4c72-a366-ce6a95578112 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.633047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b81f7-9e02-4171-9db5-743987d0d569 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.638129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849ce69-3c66-4f71-ae81-12e9ddc77b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.638887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849ce69-3c66-4f71-ae81-12e9ddc77b7e admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.639241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849ce69-3c66-4f71-ae81-12e9ddc77b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.639529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849ce69-3c66-4f71-ae81-12e9ddc77b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.640226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849ce69-3c66-4f71-ae81-12e9ddc77b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.642247 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849ce69-3c66-4f71-ae81-12e9ddc77b7e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.671170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 807/1575] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:34.672190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65b414-5239-4c72-a366-ce6a95578112 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.678034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 769/1576] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:34.680333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849ce69-3c66-4f71-ae81-12e9ddc77b7e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.750836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a65b414-5239-4c72-a366-ce6a95578112 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.757276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ebbb0-86bf-4c29-a9eb-88aad0fc5b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.758243 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ebbb0-86bf-4c29-a9eb-88aad0fc5b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.759697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ebbb0-86bf-4c29-a9eb-88aad0fc5b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.759955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ebbb0-86bf-4c29-a9eb-88aad0fc5b44 admin admin] PATH_INFO: `/v3/projects/9353d04a04ab44e58f56fd01ca43d62e/users/53f1188f2446493baae7dc79bbb66ff7/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.765440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849ce69-3c66-4f71-ae81-12e9ddc77b7e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.778455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca8db7-d771-4bb0-8af8-96fb856b50be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.778712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ebbb0-86bf-4c29-a9eb-88aad0fc5b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9353d04a04ab44e58f56fd01ca43d62e, user_id=53f1188f2446493baae7dc79bbb66ff7, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.779811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca8db7-d771-4bb0-8af8-96fb856b50be admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.780365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca8db7-d771-4bb0-8af8-96fb856b50be admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.780924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca8db7-d771-4bb0-8af8-96fb856b50be admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.781947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ebbb0-86bf-4c29-a9eb-88aad0fc5b44 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.782531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca8db7-d771-4bb0-8af8-96fb856b50be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:54:34.783055 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca8db7-d771-4bb0-8af8-96fb856b50be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.785572 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca8db7-d771-4bb0-8af8-96fb856b50be admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.801193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 770/1577] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:54:34.801962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 808/1578] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:34 2022] PUT /identity/v3/projects/9353d04a04ab44e58f56fd01ca43d62e/users/53f1188f2446493baae7dc79bbb66ff7/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:34.802809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca8db7-d771-4bb0-8af8-96fb856b50be admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.803068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ebbb0-86bf-4c29-a9eb-88aad0fc5b44 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.814196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca8db7-d771-4bb0-8af8-96fb856b50be admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.868454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:34.890985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ebbb0-86bf-4c29-a9eb-88aad0fc5b44 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.891435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee213f-082e-475e-a179-78b3d21a843c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.892147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee213f-082e-475e-a179-78b3d21a843c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.892479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee213f-082e-475e-a179-78b3d21a843c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.892752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee213f-082e-475e-a179-78b3d21a843c admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.893610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee213f-082e-475e-a179-78b3d21a843c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.895597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee213f-082e-475e-a179-78b3d21a843c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.897187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83715f34-7659-4c9b-af25-202534816f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:34.897858 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83715f34-7659-4c9b-af25-202534816f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:34.898175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83715f34-7659-4c9b-af25-202534816f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:34.898595 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83715f34-7659-4c9b-af25-202534816f6c admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:34.899337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83715f34-7659-4c9b-af25-202534816f6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:54:34.899652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83715f34-7659-4c9b-af25-202534816f6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:34.901467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83715f34-7659-4c9b-af25-202534816f6c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:34.915864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 809/1579] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:54:34.917068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83715f34-7659-4c9b-af25-202534816f6c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.929128 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83715f34-7659-4c9b-af25-202534816f6c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:34.931415 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 771/1580] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:34.935515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee213f-082e-475e-a179-78b3d21a843c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:34.995261 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:35.003813 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cf3e3-fc4e-4cd8-8f07-0304f5193a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.005371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cf3e3-fc4e-4cd8-8f07-0304f5193a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.005664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cf3e3-fc4e-4cd8-8f07-0304f5193a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.005969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cf3e3-fc4e-4cd8-8f07-0304f5193a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.006692 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cf3e3-fc4e-4cd8-8f07-0304f5193a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.008533 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676cf3e3-fc4e-4cd8-8f07-0304f5193a90 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.015575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee213f-082e-475e-a179-78b3d21a843c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.020536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 810/1581] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:35.021431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cf3e3-fc4e-4cd8-8f07-0304f5193a90 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.024060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01363f36-65c8-46a3-ac5c-b8fa23a55964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.024969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01363f36-65c8-46a3-ac5c-b8fa23a55964 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.025246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01363f36-65c8-46a3-ac5c-b8fa23a55964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.025619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01363f36-65c8-46a3-ac5c-b8fa23a55964 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.026498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01363f36-65c8-46a3-ac5c-b8fa23a55964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.028717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01363f36-65c8-46a3-ac5c-b8fa23a55964 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.033565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cf3e3-fc4e-4cd8-8f07-0304f5193a90 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.040560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790a2c7-cf46-4ac0-a20f-245b8ab79299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.041301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790a2c7-cf46-4ac0-a20f-245b8ab79299 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.041670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790a2c7-cf46-4ac0-a20f-245b8ab79299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.041989 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790a2c7-cf46-4ac0-a20f-245b8ab79299 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.042735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790a2c7-cf46-4ac0-a20f-245b8ab79299 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.045571 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b790a2c7-cf46-4ac0-a20f-245b8ab79299 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.051381 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 772/1582] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:34 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:35.052635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0600c06-e8e5-492b-b6b5-874f98752afa None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.053001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0600c06-e8e5-492b-b6b5-874f98752afa None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.053353 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0600c06-e8e5-492b-b6b5-874f98752afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.071913 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 811/1583] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:35 2022] POST /identity/v3/users => generated 290 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:35.074316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790a2c7-cf46-4ac0-a20f-245b8ab79299 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.077230 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b0600c06-e8e5-492b-b6b5-874f98752afa None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:35.078298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b0600c06-e8e5-492b-b6b5-874f98752afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:35.087293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790a2c7-cf46-4ac0-a20f-245b8ab79299 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.089242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 773/1584] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:35.089859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0600c06-e8e5-492b-b6b5-874f98752afa None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.099228 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0600c06-e8e5-492b-b6b5-874f98752afa None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.100288 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe0798-ecc6-43b3-ba9b-310c6ea33878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.102518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0798-ecc6-43b3-ba9b-310c6ea33878 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.103015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0798-ecc6-43b3-ba9b-310c6ea33878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.103745 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe0798-ecc6-43b3-ba9b-310c6ea33878 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.105043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe0798-ecc6-43b3-ba9b-310c6ea33878 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.105215 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8185245-d9e8-4a76-ba85-fc370f61b880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.105850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8185245-d9e8-4a76-ba85-fc370f61b880 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.106166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8185245-d9e8-4a76-ba85-fc370f61b880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.106448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8185245-d9e8-4a76-ba85-fc370f61b880 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.107141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8185245-d9e8-4a76-ba85-fc370f61b880 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.108385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe0798-ecc6-43b3-ba9b-310c6ea33878 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.109080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8185245-d9e8-4a76-ba85-fc370f61b880 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.140918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 812/1585] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:35 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:35.141754 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe0798-ecc6-43b3-ba9b-310c6ea33878 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.147921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 774/1586] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:35 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:35.149869 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8185245-d9e8-4a76-ba85-fc370f61b880 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.233788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe0798-ecc6-43b3-ba9b-310c6ea33878 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.234330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8185245-d9e8-4a76-ba85-fc370f61b880 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.240246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d936dc-2253-4a0b-bdca-5a6e4ce0f5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.241187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d936dc-2253-4a0b-bdca-5a6e4ce0f5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.241479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d936dc-2253-4a0b-bdca-5a6e4ce0f5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.241791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d936dc-2253-4a0b-bdca-5a6e4ce0f5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.242540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d936dc-2253-4a0b-bdca-5a6e4ce0f5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.243729 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd9838-3e04-4500-86e2-adc085af9574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.244204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d936dc-2253-4a0b-bdca-5a6e4ce0f5a8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.244420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9838-3e04-4500-86e2-adc085af9574 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.244841 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9838-3e04-4500-86e2-adc085af9574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.245115 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9838-3e04-4500-86e2-adc085af9574 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.245811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd9838-3e04-4500-86e2-adc085af9574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.247934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd9838-3e04-4500-86e2-adc085af9574 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.254969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 775/1587] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:35.257076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d936dc-2253-4a0b-bdca-5a6e4ce0f5a8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.270856 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d936dc-2253-4a0b-bdca-5a6e4ce0f5a8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.271057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 813/1588] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:35 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:35.272032 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd9838-3e04-4500-86e2-adc085af9574 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.279446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e7f2d-1474-4755-9bce-29699b7c0a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.280157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e7f2d-1474-4755-9bce-29699b7c0a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.280456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e7f2d-1474-4755-9bce-29699b7c0a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.280742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e7f2d-1474-4755-9bce-29699b7c0a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.281635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e7f2d-1474-4755-9bce-29699b7c0a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.283260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e7f2d-1474-4755-9bce-29699b7c0a03 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.284756 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd9838-3e04-4500-86e2-adc085af9574 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.290667 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be0c99-9cb3-4d1e-8173-72dc7ba17789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.291390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be0c99-9cb3-4d1e-8173-72dc7ba17789 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.291686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be0c99-9cb3-4d1e-8173-72dc7ba17789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.291970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be0c99-9cb3-4d1e-8173-72dc7ba17789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.292733 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be0c99-9cb3-4d1e-8173-72dc7ba17789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.294599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be0c99-9cb3-4d1e-8173-72dc7ba17789 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.295591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 776/1589] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:35.299336 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e7f2d-1474-4755-9bce-29699b7c0a03 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.306002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 814/1590] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:35.307195 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be0c99-9cb3-4d1e-8173-72dc7ba17789 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.314753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:35.318675 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be0c99-9cb3-4d1e-8173-72dc7ba17789 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.325056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea41e8-1d69-4a0d-bf03-a878c299867e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.325829 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea41e8-1d69-4a0d-bf03-a878c299867e admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.326204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea41e8-1d69-4a0d-bf03-a878c299867e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.326542 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea41e8-1d69-4a0d-bf03-a878c299867e admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.327552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea41e8-1d69-4a0d-bf03-a878c299867e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.329815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea41e8-1d69-4a0d-bf03-a878c299867e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.357028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 815/1591] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:35 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:35.357028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea41e8-1d69-4a0d-bf03-a878c299867e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.369158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea41e8-1d69-4a0d-bf03-a878c299867e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.373575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e7f2d-1474-4755-9bce-29699b7c0a03 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.374377 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805630ff-711b-4129-9f71-91487ef98558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.375124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805630ff-711b-4129-9f71-91487ef98558 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.375421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805630ff-711b-4129-9f71-91487ef98558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.375700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805630ff-711b-4129-9f71-91487ef98558 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.376389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805630ff-711b-4129-9f71-91487ef98558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.377963 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86359636-1138-4319-9653-b1e89104704c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.378415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805630ff-711b-4129-9f71-91487ef98558 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.378747 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86359636-1138-4319-9653-b1e89104704c service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.379129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86359636-1138-4319-9653-b1e89104704c service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.379501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86359636-1138-4319-9653-b1e89104704c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.389954 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86359636-1138-4319-9653-b1e89104704c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.391817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86359636-1138-4319-9653-b1e89104704c service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.405116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 816/1592] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:35 2022] POST /identity/v3/users => generated 302 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:35.409064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 777/1593] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:35.409064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a794f-af79-4b30-bba9-d355ca9c574c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.409064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a794f-af79-4b30-bba9-d355ca9c574c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.409064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a794f-af79-4b30-bba9-d355ca9c574c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.410818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805630ff-711b-4129-9f71-91487ef98558 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.424555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805630ff-711b-4129-9f71-91487ef98558 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.432955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cc4eb-03cd-425f-aa85-7c8dfd6ba26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.435626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cc4eb-03cd-425f-aa85-7c8dfd6ba26d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.436306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cc4eb-03cd-425f-aa85-7c8dfd6ba26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.437040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cc4eb-03cd-425f-aa85-7c8dfd6ba26d admin admin] PATH_INFO: `/v3/projects/d211354c655b469491d235e5507e61ba/users/ebfb5a6c657a4fd592ff9e67348627eb/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.444759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-842a794f-af79-4b30-bba9-d355ca9c574c None None] MFA Rules not processed for user `7b9a57f7f87748fc916386de21821c8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:35.447443 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-842a794f-af79-4b30-bba9-d355ca9c574c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:35.468547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cc4eb-03cd-425f-aa85-7c8dfd6ba26d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d211354c655b469491d235e5507e61ba, user_id=ebfb5a6c657a4fd592ff9e67348627eb, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.472310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30cc4eb-03cd-425f-aa85-7c8dfd6ba26d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.497813 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 817/1594] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:35 2022] PUT /identity/v3/projects/d211354c655b469491d235e5507e61ba/users/ebfb5a6c657a4fd592ff9e67348627eb/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:35.501804 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cc4eb-03cd-425f-aa85-7c8dfd6ba26d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.545013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=86,keystone:INSERT=14 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:35.549412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 778/1595] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:35.550500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a794f-af79-4b30-bba9-d355ca9c574c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.561628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a794f-af79-4b30-bba9-d355ca9c574c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.583811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30cc4eb-03cd-425f-aa85-7c8dfd6ba26d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.589423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980e590-3aad-461c-b01d-d4a04fd500f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.590074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980e590-3aad-461c-b01d-d4a04fd500f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.590367 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980e590-3aad-461c-b01d-d4a04fd500f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.590644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8980e590-3aad-461c-b01d-d4a04fd500f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.591409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980e590-3aad-461c-b01d-d4a04fd500f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.593035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8980e590-3aad-461c-b01d-d4a04fd500f9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.604002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 818/1596] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:35.609832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980e590-3aad-461c-b01d-d4a04fd500f9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.628370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8980e590-3aad-461c-b01d-d4a04fd500f9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.631386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8450d-9f30-4f3d-aab9-9b0ac857406c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.633046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8450d-9f30-4f3d-aab9-9b0ac857406c admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.633335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8450d-9f30-4f3d-aab9-9b0ac857406c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.633622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd8450d-9f30-4f3d-aab9-9b0ac857406c admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.634367 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8450d-9f30-4f3d-aab9-9b0ac857406c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.636061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd8450d-9f30-4f3d-aab9-9b0ac857406c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.636355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fc2c4-5fb1-452f-a77c-020adcf02fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.637311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fc2c4-5fb1-452f-a77c-020adcf02fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.638117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fc2c4-5fb1-452f-a77c-020adcf02fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.638592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fc2c4-5fb1-452f-a77c-020adcf02fd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.639941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fc2c4-5fb1-452f-a77c-020adcf02fd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.642056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fc2c4-5fb1-452f-a77c-020adcf02fd9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.651155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 779/1597] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:35.651155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8450d-9f30-4f3d-aab9-9b0ac857406c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.660723 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 819/1598] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:35.662179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fc2c4-5fb1-452f-a77c-020adcf02fd9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.669229 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd8450d-9f30-4f3d-aab9-9b0ac857406c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.674038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a401c-3726-4e18-9051-ba9d90d05548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.674858 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a401c-3726-4e18-9051-ba9d90d05548 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.675132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a401c-3726-4e18-9051-ba9d90d05548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.675482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a401c-3726-4e18-9051-ba9d90d05548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.676296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a401c-3726-4e18-9051-ba9d90d05548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.677878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fc2c4-5fb1-452f-a77c-020adcf02fd9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.678013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a401c-3726-4e18-9051-ba9d90d05548 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.684985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10989f-9926-44b3-bdb8-4b1e53fe1632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.685876 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10989f-9926-44b3-bdb8-4b1e53fe1632 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.686168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10989f-9926-44b3-bdb8-4b1e53fe1632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.686458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10989f-9926-44b3-bdb8-4b1e53fe1632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.687525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10989f-9926-44b3-bdb8-4b1e53fe1632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.689233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10989f-9926-44b3-bdb8-4b1e53fe1632 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.689477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 780/1599] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:35.693972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d331d33-ea9b-43a1-a623-fb9c94ae3f33 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.693972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d331d33-ea9b-43a1-a623-fb9c94ae3f33 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.694288 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d331d33-ea9b-43a1-a623-fb9c94ae3f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.703350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 820/1600] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:35.704065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10989f-9926-44b3-bdb8-4b1e53fe1632 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.712596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-4d331d33-ea9b-43a1-a623-fb9c94ae3f33 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:35.713855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4d331d33-ea9b-43a1-a623-fb9c94ae3f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:35.715830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10989f-9926-44b3-bdb8-4b1e53fe1632 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.721807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 781/1601] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:35.721962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720ad59-2428-42f4-850e-4bdaf62b95f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.722668 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720ad59-2428-42f4-850e-4bdaf62b95f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.723030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720ad59-2428-42f4-850e-4bdaf62b95f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.723186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720ad59-2428-42f4-850e-4bdaf62b95f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.724051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720ad59-2428-42f4-850e-4bdaf62b95f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.725619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d331d33-ea9b-43a1-a623-fb9c94ae3f33 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.734693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720ad59-2428-42f4-850e-4bdaf62b95f7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.735855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d331d33-ea9b-43a1-a623-fb9c94ae3f33 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.743889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78cd54-e106-445f-9376-7864d68c3ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.744806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78cd54-e106-445f-9376-7864d68c3ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.745148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78cd54-e106-445f-9376-7864d68c3ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.745534 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78cd54-e106-445f-9376-7864d68c3ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.746400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78cd54-e106-445f-9376-7864d68c3ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.748179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da78cd54-e106-445f-9376-7864d68c3ee1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.764266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 782/1602] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:35.765602 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78cd54-e106-445f-9376-7864d68c3ee1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.778091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78cd54-e106-445f-9376-7864d68c3ee1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.778289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 821/1603] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:35 2022] POST /identity/v3/projects => generated 403 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:35.779137 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720ad59-2428-42f4-850e-4bdaf62b95f7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.795380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:35.853408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b523485-cbbe-4cbc-a4b4-666d23c2dcfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.854782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b523485-cbbe-4cbc-a4b4-666d23c2dcfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.855373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b523485-cbbe-4cbc-a4b4-666d23c2dcfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.855856 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b523485-cbbe-4cbc-a4b4-666d23c2dcfb admin admin] PATH_INFO: `/v3/projects/d7fe1b47df3c48c3b8d069af1c78eb44/users/df627cb95dcf4706b67b34b2996250e5/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.870613 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720ad59-2428-42f4-850e-4bdaf62b95f7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.874807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b523485-cbbe-4cbc-a4b4-666d23c2dcfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7fe1b47df3c48c3b8d069af1c78eb44, user_id=df627cb95dcf4706b67b34b2996250e5, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.877730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2ad98-9ea3-45cb-b888-b297df0c1a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:35.878634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2ad98-9ea3-45cb-b888-b297df0c1a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:35.878966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2ad98-9ea3-45cb-b888-b297df0c1a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:35.879245 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2ad98-9ea3-45cb-b888-b297df0c1a57 admin admin] PATH_INFO: `/v3/projects/6c041fd4759140b6926efa8c1a4935f7/users/c3435419f72349b68c62a76638f98812/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:35.879753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b523485-cbbe-4cbc-a4b4-666d23c2dcfb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.899492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 783/1604] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:35 2022] PUT /identity/v3/projects/d7fe1b47df3c48c3b8d069af1c78eb44/users/df627cb95dcf4706b67b34b2996250e5/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:35.901076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b523485-cbbe-4cbc-a4b4-666d23c2dcfb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.907920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2ad98-9ea3-45cb-b888-b297df0c1a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c041fd4759140b6926efa8c1a4935f7, user_id=c3435419f72349b68c62a76638f98812, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:35.912472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2ad98-9ea3-45cb-b888-b297df0c1a57 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:35.929993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 822/1605] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:35 2022] PUT /identity/v3/projects/6c041fd4759140b6926efa8c1a4935f7/users/c3435419f72349b68c62a76638f98812/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:35.930610 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2ad98-9ea3-45cb-b888-b297df0c1a57 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:35.974244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b523485-cbbe-4cbc-a4b4-666d23c2dcfb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:35.998707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2ad98-9ea3-45cb-b888-b297df0c1a57 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:36.003758 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb54b5-5c14-4245-996f-22617315cfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:36.004143 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb54b5-5c14-4245-996f-22617315cfd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.004431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb54b5-5c14-4245-996f-22617315cfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.004738 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb54b5-5c14-4245-996f-22617315cfd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.005519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb54b5-5c14-4245-996f-22617315cfd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:36.007110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb54b5-5c14-4245-996f-22617315cfd7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:36.019410 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 823/1606] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:36.020414 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb54b5-5c14-4245-996f-22617315cfd7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:36.029517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:36.050943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16e56b-c784-454f-bd8c-382c207554a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:36.051627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16e56b-c784-454f-bd8c-382c207554a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.051895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16e56b-c784-454f-bd8c-382c207554a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.052173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16e56b-c784-454f-bd8c-382c207554a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.052932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16e56b-c784-454f-bd8c-382c207554a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:36.054654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16e56b-c784-454f-bd8c-382c207554a9 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:36.065511 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 784/1607] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:36.066354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16e56b-c784-454f-bd8c-382c207554a9 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:36.077198 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16e56b-c784-454f-bd8c-382c207554a9 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:36.082205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d56ee-d967-4e34-852f-a089f8840bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:36.083145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d56ee-d967-4e34-852f-a089f8840bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.083500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d56ee-d967-4e34-852f-a089f8840bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.083847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498d56ee-d967-4e34-852f-a089f8840bf0 admin admin] PATH_INFO: `/v3/projects/b4f73dfee32d43b985e9192000eaed8c/users/9cc891cc8172490394d2bfa3c8d1dcf8/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.092525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb54b5-5c14-4245-996f-22617315cfd7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:36.096687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48e0ad-f22a-47b5-8f43-d15d6529e3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:36.097660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48e0ad-f22a-47b5-8f43-d15d6529e3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.097943 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48e0ad-f22a-47b5-8f43-d15d6529e3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.098272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48e0ad-f22a-47b5-8f43-d15d6529e3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.101042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d56ee-d967-4e34-852f-a089f8840bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4f73dfee32d43b985e9192000eaed8c, user_id=9cc891cc8172490394d2bfa3c8d1dcf8, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:36.103744 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498d56ee-d967-4e34-852f-a089f8840bf0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:36.108965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48e0ad-f22a-47b5-8f43-d15d6529e3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:36.110546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48e0ad-f22a-47b5-8f43-d15d6529e3f5 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:36.118448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 785/1608] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:36 2022] PUT /identity/v3/projects/b4f73dfee32d43b985e9192000eaed8c/users/9cc891cc8172490394d2bfa3c8d1dcf8/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:36.123567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 824/1609] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:36.124338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d56ee-d967-4e34-852f-a089f8840bf0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:36.127907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371e684-69aa-4369-b9d2-987d1f65582a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.129125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371e684-69aa-4369-b9d2-987d1f65582a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.129453 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371e684-69aa-4369-b9d2-987d1f65582a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.144067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:36.179220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5371e684-69aa-4369-b9d2-987d1f65582a None None] MFA Rules not processed for user `df627cb95dcf4706b67b34b2996250e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:36.180812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5371e684-69aa-4369-b9d2-987d1f65582a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:36.228179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498d56ee-d967-4e34-852f-a089f8840bf0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:36.238548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0233e-36b4-47d6-aba3-c4286c3d0e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:36.239798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0233e-36b4-47d6-aba3-c4286c3d0e78 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.240660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0233e-36b4-47d6-aba3-c4286c3d0e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.241693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f0233e-36b4-47d6-aba3-c4286c3d0e78 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.243369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0233e-36b4-47d6-aba3-c4286c3d0e78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:36.249873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f0233e-36b4-47d6-aba3-c4286c3d0e78 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:36.278374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 825/1610] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:36.279839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d0dc3-b70c-4b83-8af0-90d08a431727 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.281009 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d0dc3-b70c-4b83-8af0-90d08a431727 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.281301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d0dc3-b70c-4b83-8af0-90d08a431727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.296473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 786/1611] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:36 2022] POST /identity/v3/users => generated 308 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:36.296473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0233e-36b4-47d6-aba3-c4286c3d0e78 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:36.302372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d91d0dc3-b70c-4b83-8af0-90d08a431727 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:36.303688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d91d0dc3-b70c-4b83-8af0-90d08a431727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:36.310589 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f0233e-36b4-47d6-aba3-c4286c3d0e78 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:36.313404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 826/1612] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:36.315748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d0dc3-b70c-4b83-8af0-90d08a431727 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:36.326351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d516e59-7663-48c1-a06e-64e7ccb34378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:36.327293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d516e59-7663-48c1-a06e-64e7ccb34378 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.327583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d516e59-7663-48c1-a06e-64e7ccb34378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.327861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d516e59-7663-48c1-a06e-64e7ccb34378 admin admin] PATH_INFO: `/v3/projects/e538a5e1474447e28c8c7d60d166e121/users/9ab7b930254b477daf4af2c2a7546268/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.331998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d0dc3-b70c-4b83-8af0-90d08a431727 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:36.338970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7756849-0c39-4d02-912a-46e8678f51a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:36.340165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7756849-0c39-4d02-912a-46e8678f51a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.340678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7756849-0c39-4d02-912a-46e8678f51a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.341081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7756849-0c39-4d02-912a-46e8678f51a0 admin admin] PATH_INFO: `/v3/projects/a1a92a7ffc864256b48a9848b27f9fd2/users/9ab84989dd384d15b316b20cc65e792a/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.353600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d516e59-7663-48c1-a06e-64e7ccb34378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e538a5e1474447e28c8c7d60d166e121, user_id=9ab7b930254b477daf4af2c2a7546268, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:36.357807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d516e59-7663-48c1-a06e-64e7ccb34378 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:36.389191 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 787/1613] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:36 2022] PUT /identity/v3/projects/e538a5e1474447e28c8c7d60d166e121/users/9ab7b930254b477daf4af2c2a7546268/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:36.390582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7756849-0c39-4d02-912a-46e8678f51a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1a92a7ffc864256b48a9848b27f9fd2, user_id=9ab84989dd384d15b316b20cc65e792a, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:36.390983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f29b9d-3872-49f2-8847-6d328856ae62 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.391274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f29b9d-3872-49f2-8847-6d328856ae62 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.391861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f29b9d-3872-49f2-8847-6d328856ae62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.393930 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7756849-0c39-4d02-912a-46e8678f51a0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:36.419076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 827/1614] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:36 2022] PUT /identity/v3/projects/a1a92a7ffc864256b48a9848b27f9fd2/users/9ab84989dd384d15b316b20cc65e792a/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:36.420446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7756849-0c39-4d02-912a-46e8678f51a0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:36.437573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-08f29b9d-3872-49f2-8847-6d328856ae62 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:36.439613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-08f29b9d-3872-49f2-8847-6d328856ae62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:36.583044 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 788/1615] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:36.584365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb84de-694d-4c6d-a887-006b2d6e49ee None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.584658 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb84de-694d-4c6d-a887-006b2d6e49ee None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.584976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb84de-694d-4c6d-a887-006b2d6e49ee None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.585334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7756849-0c39-4d02-912a-46e8678f51a0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:36.589470 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 789/1616] 10.222.0.67 () {54 vars in 961 bytes} [Thu Jun 23 11:54:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:54:36.590316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb84de-694d-4c6d-a887-006b2d6e49ee None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:36.601381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846b071-9239-4420-b36c-d17d54785d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:36.602075 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846b071-9239-4420-b36c-d17d54785d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.602363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846b071-9239-4420-b36c-d17d54785d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.602789 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846b071-9239-4420-b36c-d17d54785d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.603545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846b071-9239-4420-b36c-d17d54785d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:36.606297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846b071-9239-4420-b36c-d17d54785d59 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:36.614929 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:36.626706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 828/1617] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:36 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:36.629715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb2894-8810-4fac-88f6-ba58a68ade1c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.631165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb2894-8810-4fac-88f6-ba58a68ade1c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.631765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb2894-8810-4fac-88f6-ba58a68ade1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.672955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-15bb2894-8810-4fac-88f6-ba58a68ade1c None None] MFA Rules not processed for user `9ab7b930254b477daf4af2c2a7546268`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:36.675049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-15bb2894-8810-4fac-88f6-ba58a68ade1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:36.734323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb84de-694d-4c6d-a887-006b2d6e49ee None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:36.744128 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d990d-8a99-482b-9afc-14362e4e797f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:36.744972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d990d-8a99-482b-9afc-14362e4e797f service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.745393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d990d-8a99-482b-9afc-14362e4e797f service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.745771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d990d-8a99-482b-9afc-14362e4e797f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.766899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 829/1618] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:36.767140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d990d-8a99-482b-9afc-14362e4e797f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:36.768204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda7464-cead-4ee8-8789-1e249f9c01a2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.768499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda7464-cead-4ee8-8789-1e249f9c01a2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.768617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d990d-8a99-482b-9afc-14362e4e797f service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:36.768811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda7464-cead-4ee8-8789-1e249f9c01a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.791124 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 790/1619] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:36.792081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d990d-8a99-482b-9afc-14362e4e797f service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:36.794021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3bda7464-cead-4ee8-8789-1e249f9c01a2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:36.795634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3bda7464-cead-4ee8-8789-1e249f9c01a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:36.804786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 830/1620] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:36.806333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd441409-20cf-4298-a7ca-ac39fcc9e106 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.806631 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd441409-20cf-4298-a7ca-ac39fcc9e106 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.807168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd441409-20cf-4298-a7ca-ac39fcc9e106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.808745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d990d-8a99-482b-9afc-14362e4e797f service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:36.821129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cba617-74b7-48f0-9c84-cd202c7766ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:36.821798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cba617-74b7-48f0-9c84-cd202c7766ce service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.822214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cba617-74b7-48f0-9c84-cd202c7766ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.822503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cba617-74b7-48f0-9c84-cd202c7766ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.848527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cba617-74b7-48f0-9c84-cd202c7766ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:36.850197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cba617-74b7-48f0-9c84-cd202c7766ce service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:36.861703 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-dd441409-20cf-4298-a7ca-ac39fcc9e106 None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:36.863611 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-dd441409-20cf-4298-a7ca-ac39fcc9e106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:36.876826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 791/1621] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:36.877651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cba617-74b7-48f0-9c84-cd202c7766ce service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:36.904193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cba617-74b7-48f0-9c84-cd202c7766ce service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:36.911012 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653e75f-0db3-4858-9439-372e8c6b90e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:36.912563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653e75f-0db3-4858-9439-372e8c6b90e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:36.916010 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653e75f-0db3-4858-9439-372e8c6b90e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:36.917156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653e75f-0db3-4858-9439-372e8c6b90e0 admin admin] PATH_INFO: `/v3/projects/af99696e74ae45ddbb856f8fab22342f/users/a732034666994d7980168ec4873274a8/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:36.923598 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:36.944029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653e75f-0db3-4858-9439-372e8c6b90e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af99696e74ae45ddbb856f8fab22342f, user_id=a732034666994d7980168ec4873274a8, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:36.948143 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653e75f-0db3-4858-9439-372e8c6b90e0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:36.966643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 792/1622] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:36 2022] PUT /identity/v3/projects/af99696e74ae45ddbb856f8fab22342f/users/a732034666994d7980168ec4873274a8/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:36.969498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653e75f-0db3-4858-9439-372e8c6b90e0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:36.975101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 831/1623] 10.222.0.67 () {58 vars in 1058 bytes} [Thu Jun 23 11:54:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:36.977401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd441409-20cf-4298-a7ca-ac39fcc9e106 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:36.993192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd441409-20cf-4298-a7ca-ac39fcc9e106 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:37.072640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653e75f-0db3-4858-9439-372e8c6b90e0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:37.073566 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb31c86-aa45-45dc-a705-e1fd2dc0f8b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:37.074978 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb31c86-aa45-45dc-a705-e1fd2dc0f8b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.075412 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb31c86-aa45-45dc-a705-e1fd2dc0f8b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.076430 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb31c86-aa45-45dc-a705-e1fd2dc0f8b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.078540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea344b4-417a-431a-b47f-a3ec0e218d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:37.079527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea344b4-417a-431a-b47f-a3ec0e218d64 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.080356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea344b4-417a-431a-b47f-a3ec0e218d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.080836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea344b4-417a-431a-b47f-a3ec0e218d64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.081809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea344b4-417a-431a-b47f-a3ec0e218d64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:37.084529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea344b4-417a-431a-b47f-a3ec0e218d64 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:37.103226 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb31c86-aa45-45dc-a705-e1fd2dc0f8b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:37.108107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb31c86-aa45-45dc-a705-e1fd2dc0f8b8 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:37.129952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 832/1624] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:37.130903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb31c86-aa45-45dc-a705-e1fd2dc0f8b8 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:37.131174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 793/1625] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:36 2022] POST /identity/v3/projects => generated 405 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:37.136394 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea344b4-417a-431a-b47f-a3ec0e218d64 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:37.150268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb31c86-aa45-45dc-a705-e1fd2dc0f8b8 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:37.176087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:37.247990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb431a31-a762-47ac-a67f-3d32785a19cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:37.248922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb431a31-a762-47ac-a67f-3d32785a19cb admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.249388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb431a31-a762-47ac-a67f-3d32785a19cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.249937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb431a31-a762-47ac-a67f-3d32785a19cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.250780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb431a31-a762-47ac-a67f-3d32785a19cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:37.252926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb431a31-a762-47ac-a67f-3d32785a19cb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:37.267048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 833/1626] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:37.268125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb431a31-a762-47ac-a67f-3d32785a19cb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:37.287746 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb431a31-a762-47ac-a67f-3d32785a19cb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:37.294861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7f066-e01e-4273-8038-982161fc0196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:37.295661 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7f066-e01e-4273-8038-982161fc0196 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.295963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7f066-e01e-4273-8038-982161fc0196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.296240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7f066-e01e-4273-8038-982161fc0196 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.297229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7f066-e01e-4273-8038-982161fc0196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:37.307411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7f066-e01e-4273-8038-982161fc0196 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:37.309823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea344b4-417a-431a-b47f-a3ec0e218d64 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:37.323870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2c101-2323-4b70-8ff1-aa19bf0382ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:37.324733 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c101-2323-4b70-8ff1-aa19bf0382ea service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.325111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c101-2323-4b70-8ff1-aa19bf0382ea service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.325845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c101-2323-4b70-8ff1-aa19bf0382ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.353178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 834/1627] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:37 2022] POST /identity/v3/users => generated 309 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:37.354460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7f066-e01e-4273-8038-982161fc0196 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:37.367309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7f066-e01e-4273-8038-982161fc0196 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:37.373484 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b50ca3-3230-44b3-a6de-a7621ed1dd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:37.375158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b50ca3-3230-44b3-a6de-a7621ed1dd89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.375644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b50ca3-3230-44b3-a6de-a7621ed1dd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.377470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b50ca3-3230-44b3-a6de-a7621ed1dd89 admin admin] PATH_INFO: `/v3/projects/af99696e74ae45ddbb856f8fab22342f/users/a732034666994d7980168ec4873274a8/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.387552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b50ca3-3230-44b3-a6de-a7621ed1dd89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af99696e74ae45ddbb856f8fab22342f, user_id=a732034666994d7980168ec4873274a8, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:37.390906 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b50ca3-3230-44b3-a6de-a7621ed1dd89 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:37.419432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2c101-2323-4b70-8ff1-aa19bf0382ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:37.420448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 835/1628] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:37 2022] PUT /identity/v3/projects/af99696e74ae45ddbb856f8fab22342f/users/a732034666994d7980168ec4873274a8/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:37.422712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b50ca3-3230-44b3-a6de-a7621ed1dd89 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:37.426064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2c101-2323-4b70-8ff1-aa19bf0382ea service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:37.436723 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 794/1629] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:37.518418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b50ca3-3230-44b3-a6de-a7621ed1dd89 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:37.532451 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2694f4b-66d6-44ae-91ce-d59c2acc1a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:37.532451 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2694f4b-66d6-44ae-91ce-d59c2acc1a14 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.533057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2694f4b-66d6-44ae-91ce-d59c2acc1a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.533057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2694f4b-66d6-44ae-91ce-d59c2acc1a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.533790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2694f4b-66d6-44ae-91ce-d59c2acc1a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:37.535400 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2694f4b-66d6-44ae-91ce-d59c2acc1a14 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:37.548801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 836/1630] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:37.555393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2c101-2323-4b70-8ff1-aa19bf0382ea service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:37.567383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2c101-2323-4b70-8ff1-aa19bf0382ea service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:37.580009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842e7b72-fb90-4f0e-8eb9-b01481353e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:37.581162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e7b72-fb90-4f0e-8eb9-b01481353e62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.581446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e7b72-fb90-4f0e-8eb9-b01481353e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.581725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842e7b72-fb90-4f0e-8eb9-b01481353e62 admin admin] PATH_INFO: `/v3/projects/9c5df9a0b430488095ae75b397c2b028/users/5b4808a23aaf4122a193bbcfadd23049/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.584404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:37.617914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e7b72-fb90-4f0e-8eb9-b01481353e62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c5df9a0b430488095ae75b397c2b028, user_id=5b4808a23aaf4122a193bbcfadd23049, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:37.622179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842e7b72-fb90-4f0e-8eb9-b01481353e62 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:37.725657 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 795/1631] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:37 2022] PUT /identity/v3/projects/9c5df9a0b430488095ae75b397c2b028/users/5b4808a23aaf4122a193bbcfadd23049/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:37.727417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2694f4b-66d6-44ae-91ce-d59c2acc1a14 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:37.727417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2694f4b-66d6-44ae-91ce-d59c2acc1a14 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:37.847712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bfebe-9db7-4327-b9e4-d0efca6cf63e None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.848659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bfebe-9db7-4327-b9e4-d0efca6cf63e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.849403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bfebe-9db7-4327-b9e4-d0efca6cf63e None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.851357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 796/1632] 10.222.0.67 () {54 vars in 912 bytes} [Thu Jun 23 11:54:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:54:37.858411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c9da9-2801-4d13-ba35-b44f652c35ca None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.858965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c9da9-2801-4d13-ba35-b44f652c35ca None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.859420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c9da9-2801-4d13-ba35-b44f652c35ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.870320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed9a3a-a339-45a8-8d1d-ae879e4b6315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:37.871782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed9a3a-a339-45a8-8d1d-ae879e4b6315 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.872294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed9a3a-a339-45a8-8d1d-ae879e4b6315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.873101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed9a3a-a339-45a8-8d1d-ae879e4b6315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.874438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed9a3a-a339-45a8-8d1d-ae879e4b6315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:37.877353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed9a3a-a339-45a8-8d1d-ae879e4b6315 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:37.886627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-368c9da9-2801-4d13-ba35-b44f652c35ca None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:37.887996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-368c9da9-2801-4d13-ba35-b44f652c35ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:37.891202 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 837/1633] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:37.897095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed9a3a-a339-45a8-8d1d-ae879e4b6315 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:37.909501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed9a3a-a339-45a8-8d1d-ae879e4b6315 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:37.915983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e424651-8b07-440c-9070-b0eb8f3cf30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:37.917068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e424651-8b07-440c-9070-b0eb8f3cf30d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.917448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e424651-8b07-440c-9070-b0eb8f3cf30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.918187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e424651-8b07-440c-9070-b0eb8f3cf30d admin admin] PATH_INFO: `/v3/projects/9c5df9a0b430488095ae75b397c2b028/users/5b4808a23aaf4122a193bbcfadd23049/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.920645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e424651-8b07-440c-9070-b0eb8f3cf30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c5df9a0b430488095ae75b397c2b028, user_id=5b4808a23aaf4122a193bbcfadd23049, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:37.923587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e424651-8b07-440c-9070-b0eb8f3cf30d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:37.937442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 838/1634] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:37 2022] PUT /identity/v3/projects/9c5df9a0b430488095ae75b397c2b028/users/5b4808a23aaf4122a193bbcfadd23049/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:37.949780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501efa7-5791-4682-aeec-e06ab258f0cb None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:37.950138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501efa7-5791-4682-aeec-e06ab258f0cb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:37.950432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b501efa7-5791-4682-aeec-e06ab258f0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:37.959319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:38.007707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b501efa7-5791-4682-aeec-e06ab258f0cb None None] MFA Rules not processed for user `5b4808a23aaf4122a193bbcfadd23049`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:38.010247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b501efa7-5791-4682-aeec-e06ab258f0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:38.024422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 797/1635] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:54:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:38.030135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c9da9-2801-4d13-ba35-b44f652c35ca None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:38.188085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c9da9-2801-4d13-ba35-b44f652c35ca None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:38.215709 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 839/1636] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:37 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:38.225769 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501efa7-5791-4682-aeec-e06ab258f0cb None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:38.329187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c8029-ac5f-40e9-a664-4bebe7044eda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:38.330291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c8029-ac5f-40e9-a664-4bebe7044eda service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:38.331585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c8029-ac5f-40e9-a664-4bebe7044eda service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:38.332176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c8029-ac5f-40e9-a664-4bebe7044eda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:38.365133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b501efa7-5791-4682-aeec-e06ab258f0cb None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:38.369889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672439ee-805c-4a6a-b747-bacade58e7a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:38.370558 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672439ee-805c-4a6a-b747-bacade58e7a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:38.370844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672439ee-805c-4a6a-b747-bacade58e7a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:38.371125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672439ee-805c-4a6a-b747-bacade58e7a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:38.383079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672439ee-805c-4a6a-b747-bacade58e7a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:38.385498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672439ee-805c-4a6a-b747-bacade58e7a5 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:38.398860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 840/1637] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:38 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:38.402163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:38.461816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c8029-ac5f-40e9-a664-4bebe7044eda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:38.463391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c8029-ac5f-40e9-a664-4bebe7044eda service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:38.482899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 798/1638] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:38.698831 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672439ee-805c-4a6a-b747-bacade58e7a5 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:38.869816 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672439ee-805c-4a6a-b747-bacade58e7a5 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:38.884447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:38.885575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:38.885884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:38.886142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] PATH_INFO: `/v3/users/9ab7b930254b477daf4af2c2a7546268` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:38.889542 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab7b930254b477daf4af2c2a7546268)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:38.890647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: WARNING py.warnings [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:54:38.890647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: warnings.warn(msg) Jun 23 11:54:38.890647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: Jun 23 11:54:38.895670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:39.018778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:39.064692 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab7b930254b477daf4af2c2a7546268'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:39.067271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab7b930254b477daf4af2c2a7546268'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:39.070679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab7b930254b477daf4af2c2a7546268'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:39.122310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab7b930254b477daf4af2c2a7546268'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:39.157830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab7b930254b477daf4af2c2a7546268'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:39.189694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 841/1639] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:54:38 2022] DELETE /identity/v3/users/9ab7b930254b477daf4af2c2a7546268 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:39.191690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec5df4-a50a-41dd-804c-e030930b715c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:39.192147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec5df4-a50a-41dd-804c-e030930b715c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:39.192634 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec5df4-a50a-41dd-804c-e030930b715c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:39.225294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-28ec5df4-a50a-41dd-804c-e030930b715c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:39.227199 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-28ec5df4-a50a-41dd-804c-e030930b715c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:39.432654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 799/1640] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:39.561865 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:39.563570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:39.694687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f756925f-0729-493c-92f0-81a84cf2d8d2 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:39.703584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c200c7-ec52-4367-a433-327996e1c1e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:39.705637 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c200c7-ec52-4367-a433-327996e1c1e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:39.717506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c200c7-ec52-4367-a433-327996e1c1e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:39.718315 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c200c7-ec52-4367-a433-327996e1c1e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:39.761697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c200c7-ec52-4367-a433-327996e1c1e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:39.768717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c200c7-ec52-4367-a433-327996e1c1e0 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:39.786244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 842/1641] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:39.860321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9810ca1-4a7e-4e53-913d-dbb0c3b8d5b4 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:39.861295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9810ca1-4a7e-4e53-913d-dbb0c3b8d5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:39.861765 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9810ca1-4a7e-4e53-913d-dbb0c3b8d5b4 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:39.864759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 800/1642] 10.222.0.67 () {54 vars in 961 bytes} [Thu Jun 23 11:54:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:54:39.873432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a347959-624c-4e15-ad83-0b80b0a8e9ea None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:39.873826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a347959-624c-4e15-ad83-0b80b0a8e9ea None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:39.874207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a347959-624c-4e15-ad83-0b80b0a8e9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:39.924062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8a347959-624c-4e15-ad83-0b80b0a8e9ea None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:39.926135 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8a347959-624c-4e15-ad83-0b80b0a8e9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:40.157460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 843/1643] 10.222.0.67 () {58 vars in 1058 bytes} [Thu Jun 23 11:54:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:54:40.163727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f2fb0-129b-48a9-a75e-56515ce39990 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:40.164393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f2fb0-129b-48a9-a75e-56515ce39990 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:40.164951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f2fb0-129b-48a9-a75e-56515ce39990 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:40.167080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 801/1644] 10.222.0.67 () {54 vars in 912 bytes} [Thu Jun 23 11:54:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:54:40.173180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b872183-2d7c-49a7-a879-af412a270d1b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:40.173840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b872183-2d7c-49a7-a879-af412a270d1b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:40.174342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b872183-2d7c-49a7-a879-af412a270d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:40.221592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7b872183-2d7c-49a7-a879-af412a270d1b None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:40.223211 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7b872183-2d7c-49a7-a879-af412a270d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:40.248874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 844/1645] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:54:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:40.263541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f2fb0-129b-48a9-a75e-56515ce39990 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:40.269669 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-df6f2fb0-129b-48a9-a75e-56515ce39990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:40.451866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f2fb0-129b-48a9-a75e-56515ce39990 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:40.461833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1854a3e-c877-40f8-af0c-d7046ec34f4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:40.463086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1854a3e-c877-40f8-af0c-d7046ec34f4b service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:40.463458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1854a3e-c877-40f8-af0c-d7046ec34f4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:40.463780 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1854a3e-c877-40f8-af0c-d7046ec34f4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:40.484784 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1854a3e-c877-40f8-af0c-d7046ec34f4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:40.486737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1854a3e-c877-40f8-af0c-d7046ec34f4b service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:40.511880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 802/1646] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:41.103142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31862341-2555-4c7b-98c4-a3f7dfbe023a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:41.103592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31862341-2555-4c7b-98c4-a3f7dfbe023a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:41.103800 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31862341-2555-4c7b-98c4-a3f7dfbe023a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:41.155937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-31862341-2555-4c7b-98c4-a3f7dfbe023a None None] MFA Rules not processed for user `a732034666994d7980168ec4873274a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:41.157291 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-31862341-2555-4c7b-98c4-a3f7dfbe023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:41.223577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:41.302637 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 845/1647] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:42.047348 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1854a3e-c877-40f8-af0c-d7046ec34f4b service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:42.048879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e1854a3e-c877-40f8-af0c-d7046ec34f4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:42.071538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1854a3e-c877-40f8-af0c-d7046ec34f4b service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:42.079026 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:42.079913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:42.080383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:42.080686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] PATH_INFO: `/v3/projects/e538a5e1474447e28c8c7d60d166e121` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:42.109930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e538a5e1474447e28c8c7d60d166e121)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:42.109930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: WARNING py.warnings [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 11:54:42.109930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: warnings.warn(msg) Jun 23 11:54:42.109930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: Jun 23 11:54:42.116003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31862341-2555-4c7b-98c4-a3f7dfbe023a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:42.116003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-31862341-2555-4c7b-98c4-a3f7dfbe023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:42.120666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:42.142917 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31862341-2555-4c7b-98c4-a3f7dfbe023a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:42.164625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1e039-94fc-4719-8e3c-7c5df3c6ebe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:42.168348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1e039-94fc-4719-8e3c-7c5df3c6ebe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:42.170058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1e039-94fc-4719-8e3c-7c5df3c6ebe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:42.170623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1e039-94fc-4719-8e3c-7c5df3c6ebe0 admin admin] PATH_INFO: `/v3/users/df627cb95dcf4706b67b34b2996250e5` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:42.173320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1e039-94fc-4719-8e3c-7c5df3c6ebe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df627cb95dcf4706b67b34b2996250e5)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:42.180904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1e039-94fc-4719-8e3c-7c5df3c6ebe0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:42.224888 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e538a5e1474447e28c8c7d60d166e121', 'request_id': 'req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.243012 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e538a5e1474447e28c8c7d60d166e121', 'request_id': 'req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.271047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e538a5e1474447e28c8c7d60d166e121', 'request_id': 'req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.285290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:42.299853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] The token cache is being invalidate because project e538a5e1474447e28c8c7d60d166e121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:54:42.300258 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.301885 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.309446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 803/1648] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:54:42 2022] DELETE /identity/v3/projects/e538a5e1474447e28c8c7d60d166e121 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:42.315000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:42.323067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:42.325811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-07b1e039-94fc-4719-8e3c-7c5df3c6ebe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df627cb95dcf4706b67b34b2996250e5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.326933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-07b1e039-94fc-4719-8e3c-7c5df3c6ebe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df627cb95dcf4706b67b34b2996250e5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.327787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-07b1e039-94fc-4719-8e3c-7c5df3c6ebe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df627cb95dcf4706b67b34b2996250e5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.347788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-07b1e039-94fc-4719-8e3c-7c5df3c6ebe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df627cb95dcf4706b67b34b2996250e5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.357213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-07b1e039-94fc-4719-8e3c-7c5df3c6ebe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df627cb95dcf4706b67b34b2996250e5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.389871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 846/1649] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:54:42 2022] DELETE /identity/v3/users/df627cb95dcf4706b67b34b2996250e5 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:42.395574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152b69a-22fc-41d3-9a60-1241917a9485 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:42.397740 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152b69a-22fc-41d3-9a60-1241917a9485 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:42.399035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152b69a-22fc-41d3-9a60-1241917a9485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:42.424532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2152b69a-22fc-41d3-9a60-1241917a9485 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:42.426256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2152b69a-22fc-41d3-9a60-1241917a9485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:42.478526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf4d7b-e964-4527-8f83-1c8c3e4bb114 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:42.568213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 847/1650] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:42.599211 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152b69a-22fc-41d3-9a60-1241917a9485 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:42.602161 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2152b69a-22fc-41d3-9a60-1241917a9485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:42.606529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:42.608129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:42.608590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:42.609077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] PATH_INFO: `/v3/users/5b4808a23aaf4122a193bbcfadd23049` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:42.610474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b4808a23aaf4122a193bbcfadd23049)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:42.612579 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:42.688587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:42.711319 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b4808a23aaf4122a193bbcfadd23049'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.712651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b4808a23aaf4122a193bbcfadd23049'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.714707 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b4808a23aaf4122a193bbcfadd23049'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.725647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152b69a-22fc-41d3-9a60-1241917a9485 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:42.731852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5bf7ee-4598-4919-8e4b-42afeecbfea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:42.732846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bf7ee-4598-4919-8e4b-42afeecbfea5 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:42.733238 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bf7ee-4598-4919-8e4b-42afeecbfea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:42.733618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5bf7ee-4598-4919-8e4b-42afeecbfea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:42.736210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-be5bf7ee-4598-4919-8e4b-42afeecbfea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:42.748695 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b4808a23aaf4122a193bbcfadd23049'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.754539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5bf7ee-4598-4919-8e4b-42afeecbfea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:42.756850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5bf7ee-4598-4919-8e4b-42afeecbfea5 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:42.764664 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b4808a23aaf4122a193bbcfadd23049'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:42.779190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 848/1651] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:54:42.794884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 804/1652] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:54:42 2022] DELETE /identity/v3/users/5b4808a23aaf4122a193bbcfadd23049 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:44.569504 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11595c9d-6a55-4f13-836b-f3f21eaedc60 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:44.570530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11595c9d-6a55-4f13-836b-f3f21eaedc60 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:44.571046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11595c9d-6a55-4f13-836b-f3f21eaedc60 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:44.573877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 849/1653] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 11:54:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:54:44.612512 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:44.614076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:44.661047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08747119-2b47-48bb-96f0-9c7d484b4016 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:44.661494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08747119-2b47-48bb-96f0-9c7d484b4016 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:44.661886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08747119-2b47-48bb-96f0-9c7d484b4016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:44.696621 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:44.714391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-08747119-2b47-48bb-96f0-9c7d484b4016 None None] MFA Rules not processed for user `ee971347d23040d3984f961025e8157e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:44.720225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-08747119-2b47-48bb-96f0-9c7d484b4016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:44.843033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76755ba1-fca5-43fc-bb8c-5277a4beb0c0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:44.854682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df2551-984f-4b42-9f00-2bb11e3cd4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:44.855580 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df2551-984f-4b42-9f00-2bb11e3cd4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:44.855871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df2551-984f-4b42-9f00-2bb11e3cd4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:44.856151 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df2551-984f-4b42-9f00-2bb11e3cd4ec admin admin] PATH_INFO: `/v3/projects/9c5df9a0b430488095ae75b397c2b028` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:44.861540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df2551-984f-4b42-9f00-2bb11e3cd4ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c5df9a0b430488095ae75b397c2b028)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:44.863700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df2551-984f-4b42-9f00-2bb11e3cd4ec admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:44.925085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 850/1654] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 11:54:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:45.029834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089a346-700d-4e48-8a5d-ef77087de128 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:45.030173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089a346-700d-4e48-8a5d-ef77087de128 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:45.030458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5089a346-700d-4e48-8a5d-ef77087de128 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:45.032318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 851/1655] 10.222.0.67 () {54 vars in 960 bytes} [Thu Jun 23 11:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:54:45.057995 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015798d1-00fe-4b9d-bc27-7cee884bfd6b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:45.059199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015798d1-00fe-4b9d-bc27-7cee884bfd6b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:45.059495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015798d1-00fe-4b9d-bc27-7cee884bfd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:45.071204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-77df2551-984f-4b42-9f00-2bb11e3cd4ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c5df9a0b430488095ae75b397c2b028', 'request_id': 'req-77df2551-984f-4b42-9f00-2bb11e3cd4ec'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:45.097423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-77df2551-984f-4b42-9f00-2bb11e3cd4ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c5df9a0b430488095ae75b397c2b028', 'request_id': 'req-77df2551-984f-4b42-9f00-2bb11e3cd4ec'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:45.107523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-77df2551-984f-4b42-9f00-2bb11e3cd4ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c5df9a0b430488095ae75b397c2b028', 'request_id': 'req-77df2551-984f-4b42-9f00-2bb11e3cd4ec'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:45.110829 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-015798d1-00fe-4b9d-bc27-7cee884bfd6b None None] MFA Rules not processed for user `ee971347d23040d3984f961025e8157e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:45.112174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-015798d1-00fe-4b9d-bc27-7cee884bfd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:45.134188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-77df2551-984f-4b42-9f00-2bb11e3cd4ec admin admin] The token cache is being invalidate because project 9c5df9a0b430488095ae75b397c2b028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:54:45.135769 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-77df2551-984f-4b42-9f00-2bb11e3cd4ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:45.137590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-77df2551-984f-4b42-9f00-2bb11e3cd4ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:45.146222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 805/1656] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:54:44 2022] DELETE /identity/v3/projects/9c5df9a0b430488095ae75b397c2b028 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:45.150830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365efae-411e-4ebd-a427-d85b604d2cce None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:45.152334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365efae-411e-4ebd-a427-d85b604d2cce None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:45.152628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365efae-411e-4ebd-a427-d85b604d2cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:45.194189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1365efae-411e-4ebd-a427-d85b604d2cce None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:45.196176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1365efae-411e-4ebd-a427-d85b604d2cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:45.276432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 852/1657] 10.222.0.67 () {58 vars in 1057 bytes} [Thu Jun 23 11:54:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:45.286256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015798d1-00fe-4b9d-bc27-7cee884bfd6b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:45.305194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015798d1-00fe-4b9d-bc27-7cee884bfd6b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:45.334010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b1e63-b9c8-48d2-8a67-b81660893a53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:45.334694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b1e63-b9c8-48d2-8a67-b81660893a53 service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:45.335009 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b1e63-b9c8-48d2-8a67-b81660893a53 service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:45.335293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b1e63-b9c8-48d2-8a67-b81660893a53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:45.336171 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 806/1658] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:45.338369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d51b1e63-b9c8-48d2-8a67-b81660893a53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:45.340512 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995e77b-11f5-472d-92fc-758b4e739444 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:45.340923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995e77b-11f5-472d-92fc-758b4e739444 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:45.341262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2995e77b-11f5-472d-92fc-758b4e739444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:45.378625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b1e63-b9c8-48d2-8a67-b81660893a53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:45.380418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b1e63-b9c8-48d2-8a67-b81660893a53 service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:45.400282 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2995e77b-11f5-472d-92fc-758b4e739444 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:45.401767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2995e77b-11f5-472d-92fc-758b4e739444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:45.431759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 853/1659] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:54:45.433390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b1e63-b9c8-48d2-8a67-b81660893a53 service placement] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:45.438004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 807/1660] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:45.460199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b1e63-b9c8-48d2-8a67-b81660893a53 service placement] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:45.475229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab57c98-c065-496d-8985-ad86692edc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:45.475930 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab57c98-c065-496d-8985-ad86692edc85 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:45.476259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab57c98-c065-496d-8985-ad86692edc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:45.476553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab57c98-c065-496d-8985-ad86692edc85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:45.477371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab57c98-c065-496d-8985-ad86692edc85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:54:45.477688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab57c98-c065-496d-8985-ad86692edc85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:45.479339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab57c98-c065-496d-8985-ad86692edc85 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:45.508811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 854/1661] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:54:45.514026 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995e77b-11f5-472d-92fc-758b4e739444 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:45.526420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2995e77b-11f5-472d-92fc-758b4e739444 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:45.546636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9392bb-ac84-4244-8115-4fd88e4e7061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:45.547327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9392bb-ac84-4244-8115-4fd88e4e7061 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:45.547615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9392bb-ac84-4244-8115-4fd88e4e7061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:45.547989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9392bb-ac84-4244-8115-4fd88e4e7061 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:45.548803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9392bb-ac84-4244-8115-4fd88e4e7061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:45.550828 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9392bb-ac84-4244-8115-4fd88e4e7061 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:45.612528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 808/1662] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:45 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:45.617298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab57c98-c065-496d-8985-ad86692edc85 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:45.629648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab57c98-c065-496d-8985-ad86692edc85 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:45.707418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:45.729442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a4ce7-2577-4031-bfd8-561ab49530e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:45.742641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a4ce7-2577-4031-bfd8-561ab49530e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:45.742954 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a4ce7-2577-4031-bfd8-561ab49530e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:45.743239 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a4ce7-2577-4031-bfd8-561ab49530e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:45.744298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a4ce7-2577-4031-bfd8-561ab49530e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:45.747044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a4ce7-2577-4031-bfd8-561ab49530e0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:45.798280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 855/1663] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:45 2022] POST /identity/v3/users => generated 284 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:45.803466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9392bb-ac84-4244-8115-4fd88e4e7061 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:45.818137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9392bb-ac84-4244-8115-4fd88e4e7061 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:45.830525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3c43b-3e51-4ec2-aea3-f23e82101766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:45.831904 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3c43b-3e51-4ec2-aea3-f23e82101766 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:45.832274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3c43b-3e51-4ec2-aea3-f23e82101766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:45.832630 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb3c43b-3e51-4ec2-aea3-f23e82101766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:45.833524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3c43b-3e51-4ec2-aea3-f23e82101766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:45.835243 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb3c43b-3e51-4ec2-aea3-f23e82101766 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:45.853408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 809/1664] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:45.854973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a4ce7-2577-4031-bfd8-561ab49530e0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:45.870124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a4ce7-2577-4031-bfd8-561ab49530e0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:45.875752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012a84d-b563-49b8-b614-1906a2147781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:45.876617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012a84d-b563-49b8-b614-1906a2147781 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:45.877373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012a84d-b563-49b8-b614-1906a2147781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:45.877764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012a84d-b563-49b8-b614-1906a2147781 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:45.879036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012a84d-b563-49b8-b614-1906a2147781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:45.880868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012a84d-b563-49b8-b614-1906a2147781 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:45.897265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 856/1665] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:45.903792 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3c43b-3e51-4ec2-aea3-f23e82101766 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:45.916534 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb3c43b-3e51-4ec2-aea3-f23e82101766 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:45.923984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ded69d-bfda-458c-8c9a-160e89b80a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:45.924929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ded69d-bfda-458c-8c9a-160e89b80a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:45.925219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ded69d-bfda-458c-8c9a-160e89b80a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:45.925502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ded69d-bfda-458c-8c9a-160e89b80a5d admin admin] PATH_INFO: `/v3/projects/2217612d58a24cd09577f5875f050c14/users/91947418f2ed43bb8eccf42dc04494b2/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:45.948309 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:45.955231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ded69d-bfda-458c-8c9a-160e89b80a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2217612d58a24cd09577f5875f050c14, user_id=91947418f2ed43bb8eccf42dc04494b2, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:45.959430 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ded69d-bfda-458c-8c9a-160e89b80a5d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:45.978435 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012a84d-b563-49b8-b614-1906a2147781 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:45.980721 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5012a84d-b563-49b8-b614-1906a2147781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:46.010826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 810/1666] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:45 2022] PUT /identity/v3/projects/2217612d58a24cd09577f5875f050c14/users/91947418f2ed43bb8eccf42dc04494b2/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:46.013050 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd7d6b-0a4d-429f-9fa3-299593bde0ad None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:46.013495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd7d6b-0a4d-429f-9fa3-299593bde0ad None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:46.013963 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd7d6b-0a4d-429f-9fa3-299593bde0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:46.018101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012a84d-b563-49b8-b614-1906a2147781 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:46.112837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-31fd7d6b-0a4d-429f-9fa3-299593bde0ad None None] MFA Rules not processed for user `91947418f2ed43bb8eccf42dc04494b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:46.114739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-31fd7d6b-0a4d-429f-9fa3-299593bde0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:46.150417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771a15f-2bec-44ce-8fc5-beb4526d175f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:46.151313 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771a15f-2bec-44ce-8fc5-beb4526d175f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:46.151614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771a15f-2bec-44ce-8fc5-beb4526d175f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:46.152512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771a15f-2bec-44ce-8fc5-beb4526d175f admin admin] PATH_INFO: `/v3/projects/d7fe1b47df3c48c3b8d069af1c78eb44` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:46.156710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771a15f-2bec-44ce-8fc5-beb4526d175f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7fe1b47df3c48c3b8d069af1c78eb44)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:46.170104 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771a15f-2bec-44ce-8fc5-beb4526d175f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:46.305359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2771a15f-2bec-44ce-8fc5-beb4526d175f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7fe1b47df3c48c3b8d069af1c78eb44', 'request_id': 'req-2771a15f-2bec-44ce-8fc5-beb4526d175f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:46.332335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2771a15f-2bec-44ce-8fc5-beb4526d175f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7fe1b47df3c48c3b8d069af1c78eb44', 'request_id': 'req-2771a15f-2bec-44ce-8fc5-beb4526d175f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:46.334057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 811/1667] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:46.337571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd7d6b-0a4d-429f-9fa3-299593bde0ad None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:46.342895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2771a15f-2bec-44ce-8fc5-beb4526d175f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7fe1b47df3c48c3b8d069af1c78eb44', 'request_id': 'req-2771a15f-2bec-44ce-8fc5-beb4526d175f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:46.367010 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd7d6b-0a4d-429f-9fa3-299593bde0ad None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:46.377327 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2771a15f-2bec-44ce-8fc5-beb4526d175f admin admin] The token cache is being invalidate because project d7fe1b47df3c48c3b8d069af1c78eb44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:54:46.377629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2771a15f-2bec-44ce-8fc5-beb4526d175f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:46.378581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2771a15f-2bec-44ce-8fc5-beb4526d175f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:46.388794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 857/1668] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:54:45 2022] DELETE /identity/v3/projects/d7fe1b47df3c48c3b8d069af1c78eb44 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:54:46.392290 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caecaa5-2570-44b1-8e49-8b3449f6d824 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:46.393342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caecaa5-2570-44b1-8e49-8b3449f6d824 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:46.393561 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caecaa5-2570-44b1-8e49-8b3449f6d824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:46.433792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4caecaa5-2570-44b1-8e49-8b3449f6d824 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:46.435543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4caecaa5-2570-44b1-8e49-8b3449f6d824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:46.596593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219526c5-a878-4788-a181-6922ed7b57a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:46.597326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219526c5-a878-4788-a181-6922ed7b57a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:46.597621 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219526c5-a878-4788-a181-6922ed7b57a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:46.597904 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219526c5-a878-4788-a181-6922ed7b57a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:46.598627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219526c5-a878-4788-a181-6922ed7b57a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:46.600588 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219526c5-a878-4788-a181-6922ed7b57a9 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:46.623141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:46.668854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 858/1669] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:54:46.672865 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 812/1670] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:46 2022] POST /identity/v3/projects => generated 367 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:46.673910 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636de50d-b905-495d-9c71-9ee83a08c42f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:46.674213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636de50d-b905-495d-9c71-9ee83a08c42f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:46.674607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636de50d-b905-495d-9c71-9ee83a08c42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:46.675847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219526c5-a878-4788-a181-6922ed7b57a9 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:46.681548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-219526c5-a878-4788-a181-6922ed7b57a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:46.718425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-636de50d-b905-495d-9c71-9ee83a08c42f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:46.723908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-636de50d-b905-495d-9c71-9ee83a08c42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:46.833124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 859/1671] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:54:46.834341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636de50d-b905-495d-9c71-9ee83a08c42f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:46.846182 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636de50d-b905-495d-9c71-9ee83a08c42f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:46.850095 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219526c5-a878-4788-a181-6922ed7b57a9 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:46.854399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956c276-8ee8-4587-8627-08d213a38f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:46.855448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956c276-8ee8-4587-8627-08d213a38f4f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:46.855892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956c276-8ee8-4587-8627-08d213a38f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:46.856321 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956c276-8ee8-4587-8627-08d213a38f4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:46.857450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956c276-8ee8-4587-8627-08d213a38f4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:54:46.857918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956c276-8ee8-4587-8627-08d213a38f4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:46.858034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56333261-82e4-4519-821f-83359abcfe21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:46.858857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333261-82e4-4519-821f-83359abcfe21 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:46.859152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333261-82e4-4519-821f-83359abcfe21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:46.859439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333261-82e4-4519-821f-83359abcfe21 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:46.860164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56333261-82e4-4519-821f-83359abcfe21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:46.860445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956c276-8ee8-4587-8627-08d213a38f4f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:46.862156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56333261-82e4-4519-821f-83359abcfe21 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:46.879696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 860/1672] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:54:46.881257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956c276-8ee8-4587-8627-08d213a38f4f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:46.894363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:46.897061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956c276-8ee8-4587-8627-08d213a38f4f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:46.907201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda9c6d-8b7c-4bca-b6f0-9a23e6b40602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:46.909488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda9c6d-8b7c-4bca-b6f0-9a23e6b40602 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:46.910076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda9c6d-8b7c-4bca-b6f0-9a23e6b40602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:46.910653 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda9c6d-8b7c-4bca-b6f0-9a23e6b40602 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:46.911583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda9c6d-8b7c-4bca-b6f0-9a23e6b40602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:46.913865 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda9c6d-8b7c-4bca-b6f0-9a23e6b40602 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:46.917701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 813/1673] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:46 2022] POST /identity/v3/users => generated 290 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:54:46.918921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56333261-82e4-4519-821f-83359abcfe21 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:46.941384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56333261-82e4-4519-821f-83359abcfe21 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:46.954959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 861/1674] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:46 2022] POST /identity/v3/projects => generated 353 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:46.959438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda9c6d-8b7c-4bca-b6f0-9a23e6b40602 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:46.964059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946f507-e9cb-48c9-a6ce-9561f0db8e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:46.965116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946f507-e9cb-48c9-a6ce-9561f0db8e30 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:46.965574 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946f507-e9cb-48c9-a6ce-9561f0db8e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:46.966006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946f507-e9cb-48c9-a6ce-9561f0db8e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:46.966947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946f507-e9cb-48c9-a6ce-9561f0db8e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:46.970532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946f507-e9cb-48c9-a6ce-9561f0db8e30 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:47.012071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 814/1675] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:47.013697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946f507-e9cb-48c9-a6ce-9561f0db8e30 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:47.033316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946f507-e9cb-48c9-a6ce-9561f0db8e30 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:47.043039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda9c6d-8b7c-4bca-b6f0-9a23e6b40602 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:47.045142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c495b9-9881-4213-94d5-daa4ae70534e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:47.046261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c495b9-9881-4213-94d5-daa4ae70534e admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:47.046697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c495b9-9881-4213-94d5-daa4ae70534e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:47.047133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c495b9-9881-4213-94d5-daa4ae70534e admin admin] PATH_INFO: `/v3/projects/d0aea691d4364660ad9d1b274326f4f3/users/368bce277ad24375b9943d7e5f838f39/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:47.048533 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed33708-2043-486f-9d47-0d32768861dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:47.049278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed33708-2043-486f-9d47-0d32768861dd admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:47.049568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed33708-2043-486f-9d47-0d32768861dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:47.049863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed33708-2043-486f-9d47-0d32768861dd admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:47.050575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed33708-2043-486f-9d47-0d32768861dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:47.053422 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed33708-2043-486f-9d47-0d32768861dd admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:47.083013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c495b9-9881-4213-94d5-daa4ae70534e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0aea691d4364660ad9d1b274326f4f3, user_id=368bce277ad24375b9943d7e5f838f39, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:47.087950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c495b9-9881-4213-94d5-daa4ae70534e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:47.103464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 862/1676] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:46 2022] POST /identity/v3/users => generated 277 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:47.107961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed33708-2043-486f-9d47-0d32768861dd admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:47.113013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 815/1677] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:47 2022] PUT /identity/v3/projects/d0aea691d4364660ad9d1b274326f4f3/users/368bce277ad24375b9943d7e5f838f39/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:47.118628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c495b9-9881-4213-94d5-daa4ae70534e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:47.120936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed33708-2043-486f-9d47-0d32768861dd admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:47.191748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:47.236882 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91801c-9bba-4fd2-bf9b-f77c3c68eb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:47.237574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91801c-9bba-4fd2-bf9b-f77c3c68eb0a admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:47.237866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91801c-9bba-4fd2-bf9b-f77c3c68eb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:47.238151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91801c-9bba-4fd2-bf9b-f77c3c68eb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:47.238900 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91801c-9bba-4fd2-bf9b-f77c3c68eb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:47.241208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91801c-9bba-4fd2-bf9b-f77c3c68eb0a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:47.260558 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 863/1678] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:47.261528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91801c-9bba-4fd2-bf9b-f77c3c68eb0a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:47.275609 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91801c-9bba-4fd2-bf9b-f77c3c68eb0a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:47.281201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff94bb-7cc5-41c4-8173-50572eaad5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:47.281883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff94bb-7cc5-41c4-8173-50572eaad5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:47.282360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff94bb-7cc5-41c4-8173-50572eaad5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:47.282644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff94bb-7cc5-41c4-8173-50572eaad5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:47.283549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff94bb-7cc5-41c4-8173-50572eaad5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:47.286225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bff94bb-7cc5-41c4-8173-50572eaad5b7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:47.293477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c495b9-9881-4213-94d5-daa4ae70534e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:47.305423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 864/1679] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:47.306688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff94bb-7cc5-41c4-8173-50572eaad5b7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:47.313265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128fbe5-d29b-4139-81d9-194522da962b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:47.314334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128fbe5-d29b-4139-81d9-194522da962b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:47.314856 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128fbe5-d29b-4139-81d9-194522da962b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:47.315346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128fbe5-d29b-4139-81d9-194522da962b admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:47.316305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128fbe5-d29b-4139-81d9-194522da962b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:47.318496 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128fbe5-d29b-4139-81d9-194522da962b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:47.320554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff94bb-7cc5-41c4-8173-50572eaad5b7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:47.327336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf0113-4bac-438b-8822-024dad102ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:47.328467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf0113-4bac-438b-8822-024dad102ccf admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:47.329052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf0113-4bac-438b-8822-024dad102ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:47.329343 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf0113-4bac-438b-8822-024dad102ccf admin admin] PATH_INFO: `/v3/projects/efb0562b0b4349d190b07dce0d1a96dd/users/4730a01a041e4f13b972877f835e9f02/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:47.352911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 816/1680] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:47.357381 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128fbe5-d29b-4139-81d9-194522da962b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:47.361470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf0113-4bac-438b-8822-024dad102ccf admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb0562b0b4349d190b07dce0d1a96dd, user_id=4730a01a041e4f13b972877f835e9f02, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:47.364087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf0113-4bac-438b-8822-024dad102ccf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:47.397131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128fbe5-d29b-4139-81d9-194522da962b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:47.397345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 865/1681] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:47 2022] PUT /identity/v3/projects/efb0562b0b4349d190b07dce0d1a96dd/users/4730a01a041e4f13b972877f835e9f02/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:47.404094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605727f-e26f-4734-892c-7a7b3fb71cc6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:47.404632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605727f-e26f-4734-892c-7a7b3fb71cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:47.405056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605727f-e26f-4734-892c-7a7b3fb71cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:47.461737 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2605727f-e26f-4734-892c-7a7b3fb71cc6 None None] MFA Rules not processed for user `4730a01a041e4f13b972877f835e9f02`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:47.463175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2605727f-e26f-4734-892c-7a7b3fb71cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:47.495180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cdc7e0-4030-4358-84e2-268e58a61f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:47.496396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdc7e0-4030-4358-84e2-268e58a61f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:47.496886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdc7e0-4030-4358-84e2-268e58a61f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:47.497322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdc7e0-4030-4358-84e2-268e58a61f45 admin admin] PATH_INFO: `/v3/projects/d0aea691d4364660ad9d1b274326f4f3/users/368bce277ad24375b9943d7e5f838f39/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:47.499811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cdc7e0-4030-4358-84e2-268e58a61f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0aea691d4364660ad9d1b274326f4f3, user_id=368bce277ad24375b9943d7e5f838f39, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:47.506393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cdc7e0-4030-4358-84e2-268e58a61f45 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:47.523523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 817/1682] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:47 2022] PUT /identity/v3/projects/d0aea691d4364660ad9d1b274326f4f3/users/368bce277ad24375b9943d7e5f838f39/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:47.534570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363900-3149-4899-9a19-d2e4a9c533a9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:47.535105 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363900-3149-4899-9a19-d2e4a9c533a9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:47.535949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d363900-3149-4899-9a19-d2e4a9c533a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:47.619284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3d363900-3149-4899-9a19-d2e4a9c533a9 None None] MFA Rules not processed for user `368bce277ad24375b9943d7e5f838f39`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:47.624453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3d363900-3149-4899-9a19-d2e4a9c533a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:47.647013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 866/1683] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:47 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:47.650121 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605727f-e26f-4734-892c-7a7b3fb71cc6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:47.682794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605727f-e26f-4734-892c-7a7b3fb71cc6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:47.747468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:47.801951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 818/1684] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:47 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:47.819100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d363900-3149-4899-9a19-d2e4a9c533a9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:47.831166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3d363900-3149-4899-9a19-d2e4a9c533a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:47.859498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda70260-17ac-4171-8e3d-942e233a1c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:47.859498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda70260-17ac-4171-8e3d-942e233a1c31 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:47.859886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda70260-17ac-4171-8e3d-942e233a1c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:47.860085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda70260-17ac-4171-8e3d-942e233a1c31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:47.860837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda70260-17ac-4171-8e3d-942e233a1c31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:47.862796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda70260-17ac-4171-8e3d-942e233a1c31 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:47.931009 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 867/1685] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:47 2022] POST /identity/v3/projects => generated 355 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:47.934884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda70260-17ac-4171-8e3d-942e233a1c31 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:47.951451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d363900-3149-4899-9a19-d2e4a9c533a9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:47.995581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:48.074793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eecd1f-0209-4877-93e9-5a41082f8708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:48.075493 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eecd1f-0209-4877-93e9-5a41082f8708 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:48.076543 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eecd1f-0209-4877-93e9-5a41082f8708 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:48.076920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eecd1f-0209-4877-93e9-5a41082f8708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:48.084483 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda70260-17ac-4171-8e3d-942e233a1c31 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:48.099533 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d865e-3cf3-4d4a-a3a2-ed3c98698643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:48.100518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d865e-3cf3-4d4a-a3a2-ed3c98698643 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:48.100518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d865e-3cf3-4d4a-a3a2-ed3c98698643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:48.100842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d865e-3cf3-4d4a-a3a2-ed3c98698643 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:48.101563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d865e-3cf3-4d4a-a3a2-ed3c98698643 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:48.103543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d865e-3cf3-4d4a-a3a2-ed3c98698643 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:48.163823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 868/1686] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:47 2022] POST /identity/v3/users => generated 284 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:48.165490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d865e-3cf3-4d4a-a3a2-ed3c98698643 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:48.177691 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d865e-3cf3-4d4a-a3a2-ed3c98698643 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:48.182655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1361d54-3878-447e-a5f1-f00bcf60c9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:48.183711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1361d54-3878-447e-a5f1-f00bcf60c9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:48.184020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1361d54-3878-447e-a5f1-f00bcf60c9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:48.184304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1361d54-3878-447e-a5f1-f00bcf60c9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:48.185577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1361d54-3878-447e-a5f1-f00bcf60c9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:48.187829 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1361d54-3878-447e-a5f1-f00bcf60c9a5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:48.195872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eecd1f-0209-4877-93e9-5a41082f8708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:48.201880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eecd1f-0209-4877-93e9-5a41082f8708 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:48.209712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 869/1687] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:48.211332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1361d54-3878-447e-a5f1-f00bcf60c9a5 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:48.221861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 819/1688] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:47 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:48.237194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1361d54-3878-447e-a5f1-f00bcf60c9a5 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:48.250957 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c6d3-2476-456f-aec2-686bd0908eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:48.253684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c6d3-2476-456f-aec2-686bd0908eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:48.253987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c6d3-2476-456f-aec2-686bd0908eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:48.254322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03c6d3-2476-456f-aec2-686bd0908eb3 admin admin] PATH_INFO: `/v3/projects/ffa0c5c6339c46cbbadcb29581cbbaa0/users/d4082afd3af942c1aa17985a7b665e6c/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:48.296975 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03c6d3-2476-456f-aec2-686bd0908eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa0c5c6339c46cbbadcb29581cbbaa0, user_id=d4082afd3af942c1aa17985a7b665e6c, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:48.308166 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce03c6d3-2476-456f-aec2-686bd0908eb3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:48.315113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eecd1f-0209-4877-93e9-5a41082f8708 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:48.317432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-72eecd1f-0209-4877-93e9-5a41082f8708 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:48.338767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eecd1f-0209-4877-93e9-5a41082f8708 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:48.344481 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 870/1689] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:48 2022] PUT /identity/v3/projects/ffa0c5c6339c46cbbadcb29581cbbaa0/users/d4082afd3af942c1aa17985a7b665e6c/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:48.348617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c6d3-2476-456f-aec2-686bd0908eb3 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:48.455750 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce03c6d3-2476-456f-aec2-686bd0908eb3 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:48.461856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723050bb-9c8d-4c09-beb1-39534c5e3645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:48.462550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723050bb-9c8d-4c09-beb1-39534c5e3645 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:48.462841 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723050bb-9c8d-4c09-beb1-39534c5e3645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:48.463124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723050bb-9c8d-4c09-beb1-39534c5e3645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:48.463868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723050bb-9c8d-4c09-beb1-39534c5e3645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:48.465776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723050bb-9c8d-4c09-beb1-39534c5e3645 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:48.479939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:48.485073 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 871/1690] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:48.488446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723050bb-9c8d-4c09-beb1-39534c5e3645 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:48.503792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723050bb-9c8d-4c09-beb1-39534c5e3645 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:48.504430 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda8dcd-ee27-41ca-bdfd-e0f42a6c6fa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:48.505428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda8dcd-ee27-41ca-bdfd-e0f42a6c6fa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:48.505839 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda8dcd-ee27-41ca-bdfd-e0f42a6c6fa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:48.506423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda8dcd-ee27-41ca-bdfd-e0f42a6c6fa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:48.511602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397592d8-df70-4c5b-b4f3-0825e6299556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:48.511815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-aeda8dcd-ee27-41ca-bdfd-e0f42a6c6fa5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:48.513337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397592d8-df70-4c5b-b4f3-0825e6299556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:48.513785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397592d8-df70-4c5b-b4f3-0825e6299556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:48.514208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397592d8-df70-4c5b-b4f3-0825e6299556 admin admin] PATH_INFO: `/v3/projects/ffa0c5c6339c46cbbadcb29581cbbaa0/users/d4082afd3af942c1aa17985a7b665e6c/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:48.516698 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397592d8-df70-4c5b-b4f3-0825e6299556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa0c5c6339c46cbbadcb29581cbbaa0, user_id=d4082afd3af942c1aa17985a7b665e6c, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:48.520794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397592d8-df70-4c5b-b4f3-0825e6299556 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:48.537591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 872/1691] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:48 2022] PUT /identity/v3/projects/ffa0c5c6339c46cbbadcb29581cbbaa0/users/d4082afd3af942c1aa17985a7b665e6c/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:48.549287 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f935ee-73b7-4d7f-8671-0fb3c79df785 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:48.550398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f935ee-73b7-4d7f-8671-0fb3c79df785 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:48.550846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f935ee-73b7-4d7f-8671-0fb3c79df785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:48.595859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-00f935ee-73b7-4d7f-8671-0fb3c79df785 None None] MFA Rules not processed for user `d4082afd3af942c1aa17985a7b665e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:48.599506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-00f935ee-73b7-4d7f-8671-0fb3c79df785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:48.684928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda8dcd-ee27-41ca-bdfd-e0f42a6c6fa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:48.687584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda8dcd-ee27-41ca-bdfd-e0f42a6c6fa5 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:48.703338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:48.719350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 820/1692] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:54:48.768354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 873/1693] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:48 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:54:48.779503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda8dcd-ee27-41ca-bdfd-e0f42a6c6fa5 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:48.795489 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda8dcd-ee27-41ca-bdfd-e0f42a6c6fa5 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:48.821145 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9ecf9-651a-4bc2-80fc-000c2832cafe None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:48.821840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9ecf9-651a-4bc2-80fc-000c2832cafe None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:48.822848 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9ecf9-651a-4bc2-80fc-000c2832cafe None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:48.825803 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 874/1694] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 11:54:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:54:48.919033 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a01017-f469-43c8-8e6f-fe22598131ee None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:48.919314 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a01017-f469-43c8-8e6f-fe22598131ee None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:48.919603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a01017-f469-43c8-8e6f-fe22598131ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:48.942030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342c308-9b0d-4c31-a5e9-a61521b0c8a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:48.943085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342c308-9b0d-4c31-a5e9-a61521b0c8a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:48.943594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342c308-9b0d-4c31-a5e9-a61521b0c8a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:48.944134 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342c308-9b0d-4c31-a5e9-a61521b0c8a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:48.962732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b9a01017-f469-43c8-8e6f-fe22598131ee None None] MFA Rules not processed for user `ee971347d23040d3984f961025e8157e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:48.964019 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b9a01017-f469-43c8-8e6f-fe22598131ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:48.970113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342c308-9b0d-4c31-a5e9-a61521b0c8a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:48.972365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342c308-9b0d-4c31-a5e9-a61521b0c8a4 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:48.990965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 821/1695] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:48 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:49.238974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 875/1696] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 11:54:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:49.251338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342c308-9b0d-4c31-a5e9-a61521b0c8a4 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:49.264199 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342c308-9b0d-4c31-a5e9-a61521b0c8a4 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:49.278469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b733f21-d008-42cf-b602-1a69179ed1cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:49.279470 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b733f21-d008-42cf-b602-1a69179ed1cb service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:49.279996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b733f21-d008-42cf-b602-1a69179ed1cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:49.281228 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b733f21-d008-42cf-b602-1a69179ed1cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:49.376654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a01017-f469-43c8-8e6f-fe22598131ee None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:49.378731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b9a01017-f469-43c8-8e6f-fe22598131ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:49.408611 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a01017-f469-43c8-8e6f-fe22598131ee None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:49.412484 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b733f21-d008-42cf-b602-1a69179ed1cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:49.430293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521dc3f-b126-4f60-b253-b15254b21900 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:49.430986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521dc3f-b126-4f60-b253-b15254b21900 service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:49.431274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521dc3f-b126-4f60-b253-b15254b21900 service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:49.431559 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521dc3f-b126-4f60-b253-b15254b21900 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:49.435471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b733f21-d008-42cf-b602-1a69179ed1cb service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:49.459765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521dc3f-b126-4f60-b253-b15254b21900 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:49.461553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521dc3f-b126-4f60-b253-b15254b21900 service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:49.473124 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 822/1697] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:49 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:49.486169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b733f21-d008-42cf-b602-1a69179ed1cb service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:49.488962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8b733f21-d008-42cf-b602-1a69179ed1cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:49.490120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 876/1698] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:49.588145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:49.667839 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b733f21-d008-42cf-b602-1a69179ed1cb service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:49.680833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801e488-483f-47ca-b113-e5751344810c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:49.682262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801e488-483f-47ca-b113-e5751344810c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:49.682262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801e488-483f-47ca-b113-e5751344810c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:49.682262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801e488-483f-47ca-b113-e5751344810c admin admin] PATH_INFO: `/v3/users/7b9a57f7f87748fc916386de21821c8b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:49.684550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801e488-483f-47ca-b113-e5751344810c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b9a57f7f87748fc916386de21821c8b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:49.688578 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801e488-483f-47ca-b113-e5751344810c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:49.790154 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8801e488-483f-47ca-b113-e5751344810c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b9a57f7f87748fc916386de21821c8b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:49.791203 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8801e488-483f-47ca-b113-e5751344810c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b9a57f7f87748fc916386de21821c8b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:49.792653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8801e488-483f-47ca-b113-e5751344810c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b9a57f7f87748fc916386de21821c8b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:49.817702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8801e488-483f-47ca-b113-e5751344810c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b9a57f7f87748fc916386de21821c8b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:49.837703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8801e488-483f-47ca-b113-e5751344810c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b9a57f7f87748fc916386de21821c8b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:49.867642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 823/1699] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:54:49 2022] DELETE /identity/v3/users/7b9a57f7f87748fc916386de21821c8b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:54:50.086144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9cd36-75a1-4767-a16e-fbb3167bbf82 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:50.086598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9cd36-75a1-4767-a16e-fbb3167bbf82 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:50.086683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a9cd36-75a1-4767-a16e-fbb3167bbf82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:50.156160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d0a9cd36-75a1-4767-a16e-fbb3167bbf82 None None] MFA Rules not processed for user `53f1188f2446493baae7dc79bbb66ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:50.157458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d0a9cd36-75a1-4767-a16e-fbb3167bbf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:50.323118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 877/1700] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:50.337477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da42ce-8b32-4cbb-afff-64bc0ea05146 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:50.338393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da42ce-8b32-4cbb-afff-64bc0ea05146 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:50.338813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da42ce-8b32-4cbb-afff-64bc0ea05146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:50.380300 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b3da42ce-8b32-4cbb-afff-64bc0ea05146 None None] MFA Rules not processed for user `53f1188f2446493baae7dc79bbb66ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:50.382926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b3da42ce-8b32-4cbb-afff-64bc0ea05146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:50.413895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 824/1701] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:50.427462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9cd36-75a1-4767-a16e-fbb3167bbf82 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:50.429056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d0a9cd36-75a1-4767-a16e-fbb3167bbf82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:50.535163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:50.636254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a9cd36-75a1-4767-a16e-fbb3167bbf82 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:50.674424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9d6a3-8ffd-4137-8d2b-48765843fecb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:50.675135 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9d6a3-8ffd-4137-8d2b-48765843fecb service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:50.675550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9d6a3-8ffd-4137-8d2b-48765843fecb service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:50.675835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9d6a3-8ffd-4137-8d2b-48765843fecb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:50.701815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9d6a3-8ffd-4137-8d2b-48765843fecb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:50.703394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9d6a3-8ffd-4137-8d2b-48765843fecb service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:50.740109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 878/1702] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:51.364662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da42ce-8b32-4cbb-afff-64bc0ea05146 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:51.366448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b3da42ce-8b32-4cbb-afff-64bc0ea05146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:51.561119 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da42ce-8b32-4cbb-afff-64bc0ea05146 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:51.581388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e7b5d-f155-42c0-93c5-2ac63bde70b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:51.582702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e7b5d-f155-42c0-93c5-2ac63bde70b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:51.582938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e7b5d-f155-42c0-93c5-2ac63bde70b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:51.583232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e7b5d-f155-42c0-93c5-2ac63bde70b7 admin admin] PATH_INFO: `/v3/projects/be6f9428571347cdbe36e96b7f441522` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:51.587186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e7b5d-f155-42c0-93c5-2ac63bde70b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be6f9428571347cdbe36e96b7f441522)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:51.604436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e7b5d-f155-42c0-93c5-2ac63bde70b7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:51.831067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-917e7b5d-f155-42c0-93c5-2ac63bde70b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be6f9428571347cdbe36e96b7f441522', 'request_id': 'req-917e7b5d-f155-42c0-93c5-2ac63bde70b7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:51.846897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:51.861284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-917e7b5d-f155-42c0-93c5-2ac63bde70b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be6f9428571347cdbe36e96b7f441522', 'request_id': 'req-917e7b5d-f155-42c0-93c5-2ac63bde70b7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:51.872908 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-917e7b5d-f155-42c0-93c5-2ac63bde70b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be6f9428571347cdbe36e96b7f441522', 'request_id': 'req-917e7b5d-f155-42c0-93c5-2ac63bde70b7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:51.912170 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-917e7b5d-f155-42c0-93c5-2ac63bde70b7 admin admin] The token cache is being invalidate because project be6f9428571347cdbe36e96b7f441522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:54:51.914615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-917e7b5d-f155-42c0-93c5-2ac63bde70b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:51.916215 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-917e7b5d-f155-42c0-93c5-2ac63bde70b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:51.930461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9d6a3-8ffd-4137-8d2b-48765843fecb service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:51.931047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 825/1703] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:54:51 2022] DELETE /identity/v3/projects/be6f9428571347cdbe36e96b7f441522 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:51.949523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9d6a3-8ffd-4137-8d2b-48765843fecb service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:52.203340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bcec3a-9447-48f8-9392-c826ba17d90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:52.206604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcec3a-9447-48f8-9392-c826ba17d90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:52.206910 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcec3a-9447-48f8-9392-c826ba17d90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:52.207198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bcec3a-9447-48f8-9392-c826ba17d90e admin admin] PATH_INFO: `/v3/users/a732034666994d7980168ec4873274a8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:52.212249 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcec3a-9447-48f8-9392-c826ba17d90e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a732034666994d7980168ec4873274a8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:52.216398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bcec3a-9447-48f8-9392-c826ba17d90e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:52.342386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a7bcec3a-9447-48f8-9392-c826ba17d90e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a732034666994d7980168ec4873274a8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:52.343932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a7bcec3a-9447-48f8-9392-c826ba17d90e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a732034666994d7980168ec4873274a8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:52.345574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a7bcec3a-9447-48f8-9392-c826ba17d90e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a732034666994d7980168ec4873274a8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:52.389579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a7bcec3a-9447-48f8-9392-c826ba17d90e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a732034666994d7980168ec4873274a8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:52.416066 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a7bcec3a-9447-48f8-9392-c826ba17d90e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a732034666994d7980168ec4873274a8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:52.439755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 879/1704] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:54:51 2022] DELETE /identity/v3/users/a732034666994d7980168ec4873274a8 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:54:52.448130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a348a2-81cf-45c1-880e-7e079d5cdfeb None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:52.449078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a348a2-81cf-45c1-880e-7e079d5cdfeb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:52.449631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a348a2-81cf-45c1-880e-7e079d5cdfeb None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:52.451403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 826/1705] 10.222.0.67 () {54 vars in 912 bytes} [Thu Jun 23 11:54:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:54:52.460056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f4a5c-354f-4e4f-99ac-870cd5c798d1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:52.460415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f4a5c-354f-4e4f-99ac-870cd5c798d1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:52.460738 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214f4a5c-354f-4e4f-99ac-870cd5c798d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:52.563164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-214f4a5c-354f-4e4f-99ac-870cd5c798d1 None None] MFA Rules not processed for user `5f095d3370be457cb6687ac8f8c9f8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:52.566243 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-214f4a5c-354f-4e4f-99ac-870cd5c798d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:52.728447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:52.747932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 880/1706] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:54:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:52.831547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a348a2-81cf-45c1-880e-7e079d5cdfeb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:52.840843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b9a348a2-81cf-45c1-880e-7e079d5cdfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:52.879446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a348a2-81cf-45c1-880e-7e079d5cdfeb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:52.895220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b4ecb5-ec4e-488f-9a7b-9f896ab7854c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:52.896082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4ecb5-ec4e-488f-9a7b-9f896ab7854c service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:52.897522 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4ecb5-ec4e-488f-9a7b-9f896ab7854c service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:52.898010 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b4ecb5-ec4e-488f-9a7b-9f896ab7854c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:52.931953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4ecb5-ec4e-488f-9a7b-9f896ab7854c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:52.936577 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b4ecb5-ec4e-488f-9a7b-9f896ab7854c service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:52.963701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 827/1707] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:53.594385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f4a5c-354f-4e4f-99ac-870cd5c798d1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:53.596118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-214f4a5c-354f-4e4f-99ac-870cd5c798d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:53.779983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214f4a5c-354f-4e4f-99ac-870cd5c798d1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:53.797179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76006ed2-2a43-4ee5-afd8-caaaafb82e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:53.798198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76006ed2-2a43-4ee5-afd8-caaaafb82e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:53.798597 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76006ed2-2a43-4ee5-afd8-caaaafb82e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:53.799035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76006ed2-2a43-4ee5-afd8-caaaafb82e27 admin admin] PATH_INFO: `/v3/projects/af99696e74ae45ddbb856f8fab22342f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:53.803122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76006ed2-2a43-4ee5-afd8-caaaafb82e27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af99696e74ae45ddbb856f8fab22342f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:53.825918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76006ed2-2a43-4ee5-afd8-caaaafb82e27 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:53.930650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-76006ed2-2a43-4ee5-afd8-caaaafb82e27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af99696e74ae45ddbb856f8fab22342f', 'request_id': 'req-76006ed2-2a43-4ee5-afd8-caaaafb82e27'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:53.953797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-76006ed2-2a43-4ee5-afd8-caaaafb82e27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af99696e74ae45ddbb856f8fab22342f', 'request_id': 'req-76006ed2-2a43-4ee5-afd8-caaaafb82e27'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:53.961800 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-76006ed2-2a43-4ee5-afd8-caaaafb82e27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af99696e74ae45ddbb856f8fab22342f', 'request_id': 'req-76006ed2-2a43-4ee5-afd8-caaaafb82e27'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:53.992405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-76006ed2-2a43-4ee5-afd8-caaaafb82e27 admin admin] The token cache is being invalidate because project af99696e74ae45ddbb856f8fab22342f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:54:53.992811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-76006ed2-2a43-4ee5-afd8-caaaafb82e27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:53.993739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-76006ed2-2a43-4ee5-afd8-caaaafb82e27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:53.996708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 881/1708] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:54:53 2022] DELETE /identity/v3/projects/af99696e74ae45ddbb856f8fab22342f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:54.002190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3856f-d4ad-41a5-b988-4760abd75243 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:54.002190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3856f-d4ad-41a5-b988-4760abd75243 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:54.002659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3856f-d4ad-41a5-b988-4760abd75243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:54.042078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-74a3856f-d4ad-41a5-b988-4760abd75243 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:54.043493 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-74a3856f-d4ad-41a5-b988-4760abd75243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:54.164740 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 828/1709] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:54.172893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a5ef3-0f29-4e9e-8eef-e8da0b5b229c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:54.173234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a5ef3-0f29-4e9e-8eef-e8da0b5b229c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:54.173708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a5ef3-0f29-4e9e-8eef-e8da0b5b229c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:54.218734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2c1a5ef3-0f29-4e9e-8eef-e8da0b5b229c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:54.220122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2c1a5ef3-0f29-4e9e-8eef-e8da0b5b229c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:54.253274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 882/1710] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:54.254938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3856f-d4ad-41a5-b988-4760abd75243 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:54.268625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3856f-d4ad-41a5-b988-4760abd75243 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:54.280655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0675a-4761-4424-836f-8b1d393c7284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:54.283622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0675a-4761-4424-836f-8b1d393c7284 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:54.284111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0675a-4761-4424-836f-8b1d393c7284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:54.284965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0675a-4761-4424-836f-8b1d393c7284 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:54.289205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0675a-4761-4424-836f-8b1d393c7284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:54:54.290003 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0675a-4761-4424-836f-8b1d393c7284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:54.294139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0675a-4761-4424-836f-8b1d393c7284 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:54.324385 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 829/1711] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:54:54.329212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a5ef3-0f29-4e9e-8eef-e8da0b5b229c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:54.348511 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a5ef3-0f29-4e9e-8eef-e8da0b5b229c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:54.362211 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195797a1-5ccf-4101-9999-e066e7b33718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:54.369664 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0675a-4761-4424-836f-8b1d393c7284 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:54.370507 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195797a1-5ccf-4101-9999-e066e7b33718 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:54.371343 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195797a1-5ccf-4101-9999-e066e7b33718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:54.371656 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195797a1-5ccf-4101-9999-e066e7b33718 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:54.372928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3ce0675a-4761-4424-836f-8b1d393c7284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:54.373995 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195797a1-5ccf-4101-9999-e066e7b33718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:54.379293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195797a1-5ccf-4101-9999-e066e7b33718 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:54.395698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0675a-4761-4424-836f-8b1d393c7284 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:54.425794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada6c94-69d4-46e0-9787-1d1a10f6f9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:54.427547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada6c94-69d4-46e0-9787-1d1a10f6f9a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:54.430891 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada6c94-69d4-46e0-9787-1d1a10f6f9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:54.430891 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada6c94-69d4-46e0-9787-1d1a10f6f9a0 admin admin] PATH_INFO: `/v3/users/91947418f2ed43bb8eccf42dc04494b2` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:54.437665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada6c94-69d4-46e0-9787-1d1a10f6f9a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91947418f2ed43bb8eccf42dc04494b2)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:54.446775 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada6c94-69d4-46e0-9787-1d1a10f6f9a0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:54.449263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 883/1712] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:54 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:54.454008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195797a1-5ccf-4101-9999-e066e7b33718 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:54.546814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dada6c94-69d4-46e0-9787-1d1a10f6f9a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91947418f2ed43bb8eccf42dc04494b2'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:54.548419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dada6c94-69d4-46e0-9787-1d1a10f6f9a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91947418f2ed43bb8eccf42dc04494b2'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:54.549488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dada6c94-69d4-46e0-9787-1d1a10f6f9a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91947418f2ed43bb8eccf42dc04494b2'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:54.575939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dada6c94-69d4-46e0-9787-1d1a10f6f9a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91947418f2ed43bb8eccf42dc04494b2'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:54.592822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dada6c94-69d4-46e0-9787-1d1a10f6f9a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91947418f2ed43bb8eccf42dc04494b2'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:54.628588 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 830/1713] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:54:54 2022] DELETE /identity/v3/users/91947418f2ed43bb8eccf42dc04494b2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:54.632482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005a0e0-c593-48f0-929b-03ee83b1f991 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:54.634109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005a0e0-c593-48f0-929b-03ee83b1f991 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:54.635189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005a0e0-c593-48f0-929b-03ee83b1f991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:54.675691 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195797a1-5ccf-4101-9999-e066e7b33718 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:54.693355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:54.715568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d005a0e0-c593-48f0-929b-03ee83b1f991 None None] MFA Rules not processed for user `c3435419f72349b68c62a76638f98812`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:54.739146 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d005a0e0-c593-48f0-929b-03ee83b1f991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:54.806109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:54.896481 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbdde3-68fe-4d3c-9991-8223f21a650e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:54.897209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbdde3-68fe-4d3c-9991-8223f21a650e admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:54.898963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbdde3-68fe-4d3c-9991-8223f21a650e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:54.899264 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbdde3-68fe-4d3c-9991-8223f21a650e admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:54.901656 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbdde3-68fe-4d3c-9991-8223f21a650e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:54.907522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acbdde3-68fe-4d3c-9991-8223f21a650e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:54.977624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 831/1714] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:54.979411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fb374-8fc1-4b71-8fcb-bc24c22bf2c5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:54.979825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fb374-8fc1-4b71-8fcb-bc24c22bf2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:54.980218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fb374-8fc1-4b71-8fcb-bc24c22bf2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:55.010636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bc7fb374-8fc1-4b71-8fcb-bc24c22bf2c5 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:55.012804 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 884/1715] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:54 2022] POST /identity/v3/users => generated 287 bytes in 559 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:55.013883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bc7fb374-8fc1-4b71-8fcb-bc24c22bf2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:55.023446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3164a6-ac92-463a-b335-403f5cc46f07 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:55.023743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3164a6-ac92-463a-b335-403f5cc46f07 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:55.027450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3164a6-ac92-463a-b335-403f5cc46f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:55.031456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 832/1716] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:55.032459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fb374-8fc1-4b71-8fcb-bc24c22bf2c5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:55.038624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bc7fb374-8fc1-4b71-8fcb-bc24c22bf2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:55.080878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0a3164a6-ac92-463a-b335-403f5cc46f07 None None] MFA Rules not processed for user `c3435419f72349b68c62a76638f98812`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:55.083621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0a3164a6-ac92-463a-b335-403f5cc46f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:55.104501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 885/1717] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:55.106777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3164a6-ac92-463a-b335-403f5cc46f07 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:55.122817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0a3164a6-ac92-463a-b335-403f5cc46f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:55.176928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3164a6-ac92-463a-b335-403f5cc46f07 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:55.193895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07729414-adea-42db-8e85-89bce93ff76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:55.195738 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07729414-adea-42db-8e85-89bce93ff76c admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:55.196026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07729414-adea-42db-8e85-89bce93ff76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:55.196309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07729414-adea-42db-8e85-89bce93ff76c admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:55.197103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07729414-adea-42db-8e85-89bce93ff76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:55.198730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07729414-adea-42db-8e85-89bce93ff76c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:55.222896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 886/1718] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:55.239592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07729414-adea-42db-8e85-89bce93ff76c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:55.244013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-07729414-adea-42db-8e85-89bce93ff76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:55.248350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fb374-8fc1-4b71-8fcb-bc24c22bf2c5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:55.272537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b79a5f-243b-49d9-8371-1616077e5d05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:55.273437 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b79a5f-243b-49d9-8371-1616077e5d05 service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:55.273840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b79a5f-243b-49d9-8371-1616077e5d05 service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:55.274234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b79a5f-243b-49d9-8371-1616077e5d05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:55.277830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-18b79a5f-243b-49d9-8371-1616077e5d05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:55.308230 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b79a5f-243b-49d9-8371-1616077e5d05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:55.310247 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b79a5f-243b-49d9-8371-1616077e5d05 service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:55.335252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 833/1719] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:55.336360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b79a5f-243b-49d9-8371-1616077e5d05 service placement] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:55.340441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-18b79a5f-243b-49d9-8371-1616077e5d05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:55.493237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07729414-adea-42db-8e85-89bce93ff76c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:55.509807 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03510954-3298-44cd-ac87-2434efc23dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:55.513560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03510954-3298-44cd-ac87-2434efc23dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:55.513874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03510954-3298-44cd-ac87-2434efc23dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:55.515755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03510954-3298-44cd-ac87-2434efc23dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:55.560146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03510954-3298-44cd-ac87-2434efc23dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:55.566614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03510954-3298-44cd-ac87-2434efc23dd3 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:55.594467 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b79a5f-243b-49d9-8371-1616077e5d05 service placement] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:55.598472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 887/1720] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:55.600074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78e80c-1d61-4b01-8431-96d85101562a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:55.600478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78e80c-1d61-4b01-8431-96d85101562a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:55.600945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78e80c-1d61-4b01-8431-96d85101562a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:55.613532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48803879-48a9-4f5d-ac31-97071f623968 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:55.614235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48803879-48a9-4f5d-ac31-97071f623968 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:55.617653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48803879-48a9-4f5d-ac31-97071f623968 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:55.619116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48803879-48a9-4f5d-ac31-97071f623968 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:55.653365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1e78e80c-1d61-4b01-8431-96d85101562a None None] MFA Rules not processed for user `53f1188f2446493baae7dc79bbb66ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:55.670049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48803879-48a9-4f5d-ac31-97071f623968 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:55.672580 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48803879-48a9-4f5d-ac31-97071f623968 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:55.673977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1e78e80c-1d61-4b01-8431-96d85101562a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:55.700322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 834/1721] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:55.711098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48803879-48a9-4f5d-ac31-97071f623968 service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:55.761210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48803879-48a9-4f5d-ac31-97071f623968 service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:55.784047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5c8ed-a99d-4c92-9c3c-25e392142bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:55.785246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5c8ed-a99d-4c92-9c3c-25e392142bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:55.785739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5c8ed-a99d-4c92-9c3c-25e392142bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:55.786130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5c8ed-a99d-4c92-9c3c-25e392142bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:55.787015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5c8ed-a99d-4c92-9c3c-25e392142bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:55.789836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5c8ed-a99d-4c92-9c3c-25e392142bd3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:55.836431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 835/1722] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:55.838468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc7a9b-9c3b-4c80-82e7-e2e19f9084bc None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:55.838468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc7a9b-9c3b-4c80-82e7-e2e19f9084bc None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:55.841306 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc7a9b-9c3b-4c80-82e7-e2e19f9084bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:55.857652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 888/1723] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:55.858924 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78e80c-1d61-4b01-8431-96d85101562a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:55.874099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78e80c-1d61-4b01-8431-96d85101562a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:55.893348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7b3f1-ebf3-471c-82ae-136c91c131df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:55.896808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7b3f1-ebf3-471c-82ae-136c91c131df admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:55.898977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7b3f1-ebf3-471c-82ae-136c91c131df admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:55.901080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7b3f1-ebf3-471c-82ae-136c91c131df admin admin] PATH_INFO: `/v3/projects/c7dfd2e9802946f9801d4d6e055a79d5/users/be161ed539ab44c789c5587507ddeedb/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:55.947804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:55.961191 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-89dc7a9b-9c3b-4c80-82e7-e2e19f9084bc None None] MFA Rules not processed for user `ebfb5a6c657a4fd592ff9e67348627eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:55.964948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-89dc7a9b-9c3b-4c80-82e7-e2e19f9084bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:55.978925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7b3f1-ebf3-471c-82ae-136c91c131df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7dfd2e9802946f9801d4d6e055a79d5, user_id=be161ed539ab44c789c5587507ddeedb, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:55.999524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7b3f1-ebf3-471c-82ae-136c91c131df admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:56.025306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 889/1724] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:55 2022] PUT /identity/v3/projects/c7dfd2e9802946f9801d4d6e055a79d5/users/be161ed539ab44c789c5587507ddeedb/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:56.035715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136ab40-0739-4e10-8913-4eef9b2d7a94 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:56.042363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136ab40-0739-4e10-8913-4eef9b2d7a94 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:56.042918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136ab40-0739-4e10-8913-4eef9b2d7a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:56.070349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:56.127518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2136ab40-0739-4e10-8913-4eef9b2d7a94 None None] MFA Rules not processed for user `be161ed539ab44c789c5587507ddeedb`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:56.129680 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2136ab40-0739-4e10-8913-4eef9b2d7a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:56.323840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 836/1725] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 483 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:56.396194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 890/1726] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:56.397246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc7a9b-9c3b-4c80-82e7-e2e19f9084bc None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:56.430797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc7a9b-9c3b-4c80-82e7-e2e19f9084bc None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:56.575405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc12d2-c49c-4bdb-a91d-6fa03e125375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:56.577115 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc12d2-c49c-4bdb-a91d-6fa03e125375 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:56.577682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc12d2-c49c-4bdb-a91d-6fa03e125375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:56.578968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc12d2-c49c-4bdb-a91d-6fa03e125375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:56.580527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc12d2-c49c-4bdb-a91d-6fa03e125375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:56.585550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc12d2-c49c-4bdb-a91d-6fa03e125375 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:56.647188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 837/1727] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:54:56 2022] POST /identity/v3/projects => generated 371 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:56.651944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136ab40-0739-4e10-8913-4eef9b2d7a94 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:56.672421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136ab40-0739-4e10-8913-4eef9b2d7a94 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:56.885983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f672e-c0b4-46e1-a7ea-050619afc8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:56.886671 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f672e-c0b4-46e1-a7ea-050619afc8ee admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:56.886966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f672e-c0b4-46e1-a7ea-050619afc8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:56.887247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f672e-c0b4-46e1-a7ea-050619afc8ee admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:56.893605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f672e-c0b4-46e1-a7ea-050619afc8ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:56.896361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f672e-c0b4-46e1-a7ea-050619afc8ee admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:56.986525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 891/1728] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:54:56 2022] POST /identity/v3/users => generated 292 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:54:56.987764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc12d2-c49c-4bdb-a91d-6fa03e125375 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:57.016809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc12d2-c49c-4bdb-a91d-6fa03e125375 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:57.039001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594048c-a80a-43da-b069-bc0b70489a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:57.039782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594048c-a80a-43da-b069-bc0b70489a5a admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:57.040146 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594048c-a80a-43da-b069-bc0b70489a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:57.040581 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594048c-a80a-43da-b069-bc0b70489a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:57.042295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594048c-a80a-43da-b069-bc0b70489a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:57.044524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0594048c-a80a-43da-b069-bc0b70489a5a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:57.059790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 838/1729] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:57.061157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f672e-c0b4-46e1-a7ea-050619afc8ee admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:57.076377 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f672e-c0b4-46e1-a7ea-050619afc8ee admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:57.088936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fae64-0e88-4fe3-8c37-8f8010715178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:57.090051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fae64-0e88-4fe3-8c37-8f8010715178 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:57.090523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fae64-0e88-4fe3-8c37-8f8010715178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:57.090973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440fae64-0e88-4fe3-8c37-8f8010715178 admin admin] PATH_INFO: `/v3/projects/4c2cfa97ca86470c86f0d436da74f89d/users/d513b5b97c9048029e9bb10799f246fe/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:57.160013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fae64-0e88-4fe3-8c37-8f8010715178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c2cfa97ca86470c86f0d436da74f89d, user_id=d513b5b97c9048029e9bb10799f246fe, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:57.162929 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440fae64-0e88-4fe3-8c37-8f8010715178 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:57.187694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 892/1730] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:57 2022] PUT /identity/v3/projects/4c2cfa97ca86470c86f0d436da74f89d/users/d513b5b97c9048029e9bb10799f246fe/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:54:57.192210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594048c-a80a-43da-b069-bc0b70489a5a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:57.207401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594048c-a80a-43da-b069-bc0b70489a5a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:57.391997 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5e1a34-06bf-4674-b5f4-dcc93a1dafcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:57.392903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e1a34-06bf-4674-b5f4-dcc93a1dafcf admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:57.393125 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e1a34-06bf-4674-b5f4-dcc93a1dafcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:57.393437 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5e1a34-06bf-4674-b5f4-dcc93a1dafcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:57.394144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e1a34-06bf-4674-b5f4-dcc93a1dafcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:57.401301 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5e1a34-06bf-4674-b5f4-dcc93a1dafcf admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:57.427174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 839/1731] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:54:57.430655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fae64-0e88-4fe3-8c37-8f8010715178 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:57.456219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440fae64-0e88-4fe3-8c37-8f8010715178 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:57.476689 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f651d6-81fd-4c53-ba2e-e029cf225f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:57.477614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f651d6-81fd-4c53-ba2e-e029cf225f5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:57.477904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f651d6-81fd-4c53-ba2e-e029cf225f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:57.478202 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f651d6-81fd-4c53-ba2e-e029cf225f5c admin admin] PATH_INFO: `/v3/projects/4c2cfa97ca86470c86f0d436da74f89d/users/d513b5b97c9048029e9bb10799f246fe/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:57.490165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f651d6-81fd-4c53-ba2e-e029cf225f5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c2cfa97ca86470c86f0d436da74f89d, user_id=d513b5b97c9048029e9bb10799f246fe, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:57.494062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f651d6-81fd-4c53-ba2e-e029cf225f5c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:57.533309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 893/1732] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:54:57 2022] PUT /identity/v3/projects/4c2cfa97ca86470c86f0d436da74f89d/users/d513b5b97c9048029e9bb10799f246fe/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:54:57.545349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc3f6b-4c7d-458b-b63e-31e232fc4a99 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:57.545701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc3f6b-4c7d-458b-b63e-31e232fc4a99 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:57.545991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc3f6b-4c7d-458b-b63e-31e232fc4a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:57.596140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:57.603476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-fbdc3f6b-4c7d-458b-b63e-31e232fc4a99 None None] MFA Rules not processed for user `d513b5b97c9048029e9bb10799f246fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:57.610884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fbdc3f6b-4c7d-458b-b63e-31e232fc4a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:57.745592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e7029-47ef-4ccf-a06d-479440fa8d12 None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:57.746284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e7029-47ef-4ccf-a06d-479440fa8d12 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:57.746723 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e7029-47ef-4ccf-a06d-479440fa8d12 None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:57.750314 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 894/1733] 10.222.0.67 () {54 vars in 912 bytes} [Thu Jun 23 11:54:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:54:57.760889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b67e3e-5a1d-4e22-a3a9-445f46fd0016 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:57.761368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b67e3e-5a1d-4e22-a3a9-445f46fd0016 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:57.761841 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b67e3e-5a1d-4e22-a3a9-445f46fd0016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:57.801572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-29b67e3e-5a1d-4e22-a3a9-445f46fd0016 None None] MFA Rules not processed for user `5f095d3370be457cb6687ac8f8c9f8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:57.802934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-29b67e3e-5a1d-4e22-a3a9-445f46fd0016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:57.837436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 840/1734] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:54:57 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:57.866198 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc3f6b-4c7d-458b-b63e-31e232fc4a99 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:57.868710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fbdc3f6b-4c7d-458b-b63e-31e232fc4a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:57.930741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:57.959936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 895/1735] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:54:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:54:58.028402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc3f6b-4c7d-458b-b63e-31e232fc4a99 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:58.049618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57f5d3-d65b-44a3-9c1a-b4393493e789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:58.050339 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57f5d3-d65b-44a3-9c1a-b4393493e789 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:58.050635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57f5d3-d65b-44a3-9c1a-b4393493e789 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:58.050879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b67e3e-5a1d-4e22-a3a9-445f46fd0016 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:58.050996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57f5d3-d65b-44a3-9c1a-b4393493e789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:58.055642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-29b67e3e-5a1d-4e22-a3a9-445f46fd0016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:58.066294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57f5d3-d65b-44a3-9c1a-b4393493e789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:58.067939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57f5d3-d65b-44a3-9c1a-b4393493e789 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:58.080474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b67e3e-5a1d-4e22-a3a9-445f46fd0016 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:58.081829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 841/1736] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:54:58.091721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57f5d3-d65b-44a3-9c1a-b4393493e789 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:58.093950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fa57f5d3-d65b-44a3-9c1a-b4393493e789 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:58.125633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9b9a3-de1f-47bd-a8d8-ac87e6ef6c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:58.137394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9b9a3-de1f-47bd-a8d8-ac87e6ef6c08 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:58.137696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9b9a3-de1f-47bd-a8d8-ac87e6ef6c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:58.144168 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9b9a3-de1f-47bd-a8d8-ac87e6ef6c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:58.234175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9b9a3-de1f-47bd-a8d8-ac87e6ef6c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:58.239939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57f5d3-d65b-44a3-9c1a-b4393493e789 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:58.245346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:58.247784 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:58.247784 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:58.247784 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] PATH_INFO: `/v3/projects/2217612d58a24cd09577f5875f050c14` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:58.249238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2217612d58a24cd09577f5875f050c14)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:58.251320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:58.251763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9b9a3-de1f-47bd-a8d8-ac87e6ef6c08 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:58.305383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 896/1737] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:54:58.343152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:58.350076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2217612d58a24cd09577f5875f050c14', 'request_id': 'req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:58.370235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2217612d58a24cd09577f5875f050c14', 'request_id': 'req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:58.391465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2217612d58a24cd09577f5875f050c14', 'request_id': 'req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:58.446654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] The token cache is being invalidate because project 2217612d58a24cd09577f5875f050c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:54:58.449210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:58.449210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:58.457008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9b9a3-de1f-47bd-a8d8-ac87e6ef6c08 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:58.480701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 842/1738] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:54:58 2022] DELETE /identity/v3/projects/2217612d58a24cd09577f5875f050c14 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:58.512774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9b9a3-de1f-47bd-a8d8-ac87e6ef6c08 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:58.834873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:58.837716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:58.905713 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:58.906617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:58.922326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:58.922728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] PATH_INFO: `/v3/users/368bce277ad24375b9943d7e5f838f39` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:58.934042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=368bce277ad24375b9943d7e5f838f39)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:58.943770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:59.280782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '368bce277ad24375b9943d7e5f838f39'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:59.282379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '368bce277ad24375b9943d7e5f838f39'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:59.286146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '368bce277ad24375b9943d7e5f838f39'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:59.354425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '368bce277ad24375b9943d7e5f838f39'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:59.382993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bc2d3b-cbc7-45d5-ba0f-eb58bb75d645 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:59.386418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '368bce277ad24375b9943d7e5f838f39'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:59.403454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b88e8-38cc-4af4-b75d-5f39d726ad22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:59.411896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b88e8-38cc-4af4-b75d-5f39d726ad22 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:59.417498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b88e8-38cc-4af4-b75d-5f39d726ad22 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:59.417827 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b88e8-38cc-4af4-b75d-5f39d726ad22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:59.423643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-115b88e8-38cc-4af4-b75d-5f39d726ad22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:59.434125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 897/1739] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:54:58 2022] DELETE /identity/v3/users/368bce277ad24375b9943d7e5f838f39 => generated 0 bytes in 974 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:54:59.560774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:54:59.562380 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:59.671998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b88e8-38cc-4af4-b75d-5f39d726ad22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:59.673720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115b88e8-38cc-4af4-b75d-5f39d726ad22 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:59.709041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 843/1740] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:54:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 875 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:54:59.723285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f907a6-da53-469d-a29f-a05956336f37 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:59.723576 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f907a6-da53-469d-a29f-a05956336f37 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:59.723872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f907a6-da53-469d-a29f-a05956336f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:59.739630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b22c01-f9b3-4801-91a6-8efa413f6d9c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:54:59.752296 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ffce3-7296-4085-b454-615dbbf79143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:54:59.755179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ffce3-7296-4085-b454-615dbbf79143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:54:59.755593 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ffce3-7296-4085-b454-615dbbf79143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:54:59.757077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ffce3-7296-4085-b454-615dbbf79143 admin admin] PATH_INFO: `/v3/users/be161ed539ab44c789c5587507ddeedb` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:54:59.759711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ffce3-7296-4085-b454-615dbbf79143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be161ed539ab44c789c5587507ddeedb)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:54:59.760977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-44f907a6-da53-469d-a29f-a05956336f37 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:54:59.765849 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-44f907a6-da53-469d-a29f-a05956336f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:54:59.768520 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ffce3-7296-4085-b454-615dbbf79143 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:54:59.774068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:54:59.935373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1d8ffce3-7296-4085-b454-615dbbf79143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be161ed539ab44c789c5587507ddeedb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:59.936867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1d8ffce3-7296-4085-b454-615dbbf79143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be161ed539ab44c789c5587507ddeedb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:59.943034 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1d8ffce3-7296-4085-b454-615dbbf79143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be161ed539ab44c789c5587507ddeedb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:54:59.994103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1d8ffce3-7296-4085-b454-615dbbf79143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be161ed539ab44c789c5587507ddeedb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:00.025385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1d8ffce3-7296-4085-b454-615dbbf79143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be161ed539ab44c789c5587507ddeedb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:00.053660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 844/1741] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:54:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:00.055857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce000e-4e26-4d0b-8393-d160c93fa290 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:00.056554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce000e-4e26-4d0b-8393-d160c93fa290 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:00.065261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce000e-4e26-4d0b-8393-d160c93fa290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:00.076159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 898/1742] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:54:59 2022] DELETE /identity/v3/users/be161ed539ab44c789c5587507ddeedb => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:00.080963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c1b8f-ee34-44d3-b19b-d3ecb3f68ba3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:00.081264 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c1b8f-ee34-44d3-b19b-d3ecb3f68ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:00.081556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362c1b8f-ee34-44d3-b19b-d3ecb3f68ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:00.178479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-362c1b8f-ee34-44d3-b19b-d3ecb3f68ba3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:00.185730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:00.186044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-362c1b8f-ee34-44d3-b19b-d3ecb3f68ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:00.196809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-99ce000e-4e26-4d0b-8393-d160c93fa290 None None] MFA Rules not processed for user `9ab84989dd384d15b316b20cc65e792a`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:00.199515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-99ce000e-4e26-4d0b-8393-d160c93fa290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:00.429808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 899/1743] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:00.429808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c1b8f-ee34-44d3-b19b-d3ecb3f68ba3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:00.444313 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-362c1b8f-ee34-44d3-b19b-d3ecb3f68ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:00.481594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 845/1744] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:00.490589 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce000e-4e26-4d0b-8393-d160c93fa290 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:00.492094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-99ce000e-4e26-4d0b-8393-d160c93fa290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:00.619336 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce000e-4e26-4d0b-8393-d160c93fa290 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:00.628791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56b011-0f81-4b89-b523-88ac20231b89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:00.629481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56b011-0f81-4b89-b523-88ac20231b89 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:00.629774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56b011-0f81-4b89-b523-88ac20231b89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:00.630160 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56b011-0f81-4b89-b523-88ac20231b89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:00.661139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56b011-0f81-4b89-b523-88ac20231b89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:00.666705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56b011-0f81-4b89-b523-88ac20231b89 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:00.680988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 846/1745] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:00.681887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56b011-0f81-4b89-b523-88ac20231b89 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:00.690560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9e56b011-0f81-4b89-b523-88ac20231b89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:00.721747 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362c1b8f-ee34-44d3-b19b-d3ecb3f68ba3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:00.735815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095b288e-104f-470a-ac2d-e7685224923e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:00.736503 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b288e-104f-470a-ac2d-e7685224923e service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:00.737783 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b288e-104f-470a-ac2d-e7685224923e service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:00.738080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095b288e-104f-470a-ac2d-e7685224923e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:00.855891 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56b011-0f81-4b89-b523-88ac20231b89 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:00.859520 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:00.872526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8342ec-ec67-44ac-8857-54baf6e6353a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:00.873287 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8342ec-ec67-44ac-8857-54baf6e6353a service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:00.873532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8342ec-ec67-44ac-8857-54baf6e6353a service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:00.873819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8342ec-ec67-44ac-8857-54baf6e6353a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:00.889694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8342ec-ec67-44ac-8857-54baf6e6353a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:00.891351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8342ec-ec67-44ac-8857-54baf6e6353a service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:00.906910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 847/1746] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:01.017859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b288e-104f-470a-ac2d-e7685224923e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:01.020150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095b288e-104f-470a-ac2d-e7685224923e service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:01.032868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 900/1747] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:55:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:55:01.508105 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8342ec-ec67-44ac-8857-54baf6e6353a service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:01.509056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f09506-6e3c-414c-bec1-a86b2e55a08f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:01.510181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9b8342ec-ec67-44ac-8857-54baf6e6353a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:01.510436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f09506-6e3c-414c-bec1-a86b2e55a08f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:01.511053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f09506-6e3c-414c-bec1-a86b2e55a08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:01.588500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8342ec-ec67-44ac-8857-54baf6e6353a service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:01.591227 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d6f09506-6e3c-414c-bec1-a86b2e55a08f None None] MFA Rules not processed for user `9cc891cc8172490394d2bfa3c8d1dcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:01.592947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d6f09506-6e3c-414c-bec1-a86b2e55a08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:01.657508 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:01.661179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:01.661637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:01.682943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e admin admin] PATH_INFO: `/v3/projects/d0aea691d4364660ad9d1b274326f4f3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:01.688463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0aea691d4364660ad9d1b274326f4f3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:01.690865 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:01.760332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 901/1748] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:01.801739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f09506-6e3c-414c-bec1-a86b2e55a08f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:01.803823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d6f09506-6e3c-414c-bec1-a86b2e55a08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:01.873333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f09506-6e3c-414c-bec1-a86b2e55a08f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:01.890264 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7f643-0c5b-44de-9688-b892d041c144 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:01.892277 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:01.893635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7f643-0c5b-44de-9688-b892d041c144 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:01.895545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7f643-0c5b-44de-9688-b892d041c144 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:01.896165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7f643-0c5b-44de-9688-b892d041c144 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:01.924280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7f643-0c5b-44de-9688-b892d041c144 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:01.930437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7f643-0c5b-44de-9688-b892d041c144 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:01.955850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0aea691d4364660ad9d1b274326f4f3', 'request_id': 'req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:01.981464 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0aea691d4364660ad9d1b274326f4f3', 'request_id': 'req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:01.985116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 902/1749] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:01.998932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0aea691d4364660ad9d1b274326f4f3', 'request_id': 'req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:02.061017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e admin admin] The token cache is being invalidate because project d0aea691d4364660ad9d1b274326f4f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:02.061250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:02.064046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0fa939b1-31a6-424e-bfbd-1a5cc29eba8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:02.083194 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 848/1750] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:01 2022] DELETE /identity/v3/projects/d0aea691d4364660ad9d1b274326f4f3 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:02.084996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16525c0f-80b7-43f0-bbe0-0824e31993d7 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:02.086026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16525c0f-80b7-43f0-bbe0-0824e31993d7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:02.086118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16525c0f-80b7-43f0-bbe0-0824e31993d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:02.162893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-16525c0f-80b7-43f0-bbe0-0824e31993d7 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:02.170270 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-16525c0f-80b7-43f0-bbe0-0824e31993d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:02.480048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176561c7-6a0b-415c-97d0-58863130e4f6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:02.480558 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176561c7-6a0b-415c-97d0-58863130e4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:02.480861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 903/1751] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:02.481036 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176561c7-6a0b-415c-97d0-58863130e4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:02.548033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-176561c7-6a0b-415c-97d0-58863130e4f6 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:02.560890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-176561c7-6a0b-415c-97d0-58863130e4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:02.617247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16525c0f-80b7-43f0-bbe0-0824e31993d7 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:02.629365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 849/1752] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:02.638415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16525c0f-80b7-43f0-bbe0-0824e31993d7 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:02.692828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb8335-2471-4fed-8a2d-5dad7efcc7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:02.693086 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb8335-2471-4fed-8a2d-5dad7efcc7b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:02.693423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb8335-2471-4fed-8a2d-5dad7efcc7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:02.693655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb8335-2471-4fed-8a2d-5dad7efcc7b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:02.698326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb8335-2471-4fed-8a2d-5dad7efcc7b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:02.698425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb8335-2471-4fed-8a2d-5dad7efcc7b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:02.704998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb8335-2471-4fed-8a2d-5dad7efcc7b8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:02.783924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176561c7-6a0b-415c-97d0-58863130e4f6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:02.787418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 904/1753] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:02.821928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176561c7-6a0b-415c-97d0-58863130e4f6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:02.841048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccbf3e-6934-4e4c-b791-0e7204f9c4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:02.843870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccbf3e-6934-4e4c-b791-0e7204f9c4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:02.844340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccbf3e-6934-4e4c-b791-0e7204f9c4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:02.844821 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ccbf3e-6934-4e4c-b791-0e7204f9c4e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:02.845716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccbf3e-6934-4e4c-b791-0e7204f9c4e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:02.853845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ccbf3e-6934-4e4c-b791-0e7204f9c4e8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:02.950137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 850/1754] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:02 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:02.954386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb8335-2471-4fed-8a2d-5dad7efcc7b8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:02.972585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb8335-2471-4fed-8a2d-5dad7efcc7b8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:03.077425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:03.148342 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccbf3e-6934-4e4c-b791-0e7204f9c4e8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:03.150720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a9ccbf3e-6934-4e4c-b791-0e7204f9c4e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:03.168030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edf235-763e-442a-9e2a-e41cfd0bd99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:03.169120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edf235-763e-442a-9e2a-e41cfd0bd99e admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:03.169416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edf235-763e-442a-9e2a-e41cfd0bd99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:03.169692 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edf235-763e-442a-9e2a-e41cfd0bd99e admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:03.172260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edf235-763e-442a-9e2a-e41cfd0bd99e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:03.177654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edf235-763e-442a-9e2a-e41cfd0bd99e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:03.234367 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 905/1755] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:02 2022] POST /identity/v3/users => generated 294 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:03.235645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edf235-763e-442a-9e2a-e41cfd0bd99e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:03.265446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edf235-763e-442a-9e2a-e41cfd0bd99e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:03.302087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9acb3-87d1-4a0a-a342-612f068f1db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:03.302859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9acb3-87d1-4a0a-a342-612f068f1db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:03.303151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9acb3-87d1-4a0a-a342-612f068f1db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:03.303479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9acb3-87d1-4a0a-a342-612f068f1db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:03.304836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9acb3-87d1-4a0a-a342-612f068f1db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:03.306569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9acb3-87d1-4a0a-a342-612f068f1db1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:03.333077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ccbf3e-6934-4e4c-b791-0e7204f9c4e8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:03.357712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:03.360028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:03.360889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:03.361540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] PATH_INFO: `/v3/projects/c7dfd2e9802946f9801d4d6e055a79d5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:03.364312 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7dfd2e9802946f9801d4d6e055a79d5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:03.367622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:03.391717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 906/1756] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:03.392857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9acb3-87d1-4a0a-a342-612f068f1db1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:03.435779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9acb3-87d1-4a0a-a342-612f068f1db1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:03.460792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12017428-1e64-4633-b8f1-81e545ef288f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:03.464860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12017428-1e64-4633-b8f1-81e545ef288f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:03.465157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12017428-1e64-4633-b8f1-81e545ef288f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:03.465520 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12017428-1e64-4633-b8f1-81e545ef288f admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:03.466394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12017428-1e64-4633-b8f1-81e545ef288f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:03.467994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12017428-1e64-4633-b8f1-81e545ef288f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:03.517815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7dfd2e9802946f9801d4d6e055a79d5', 'request_id': 'req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:03.520223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 907/1757] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:03.521239 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12017428-1e64-4633-b8f1-81e545ef288f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:03.540259 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7dfd2e9802946f9801d4d6e055a79d5', 'request_id': 'req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:03.554147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12017428-1e64-4633-b8f1-81e545ef288f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:03.555323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7dfd2e9802946f9801d4d6e055a79d5', 'request_id': 'req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:03.580895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] The token cache is being invalidate because project c7dfd2e9802946f9801d4d6e055a79d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:03.581194 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:03.583984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:03.590782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 851/1758] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:03 2022] DELETE /identity/v3/projects/c7dfd2e9802946f9801d4d6e055a79d5 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:03.594013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:03.598316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:03.647358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:03.766560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c85b3-eaa3-4be4-ba3d-aea4880236fb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:03.780949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:03.782187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:03.782627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:03.783056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] PATH_INFO: `/v3/users/d513b5b97c9048029e9bb10799f246fe` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:03.786414 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d513b5b97c9048029e9bb10799f246fe)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:03.792458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:03.802596 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31acb118-86b4-41eb-bf6f-3ef9697b47f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:03.807971 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acb118-86b4-41eb-bf6f-3ef9697b47f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:03.808624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acb118-86b4-41eb-bf6f-3ef9697b47f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:03.809180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31acb118-86b4-41eb-bf6f-3ef9697b47f6 admin admin] PATH_INFO: `/v3/projects/d73908ad335745ab842ca8e8b91924f7/users/e99835e9f0a24ba6a941814f820a61d6/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:03.870235 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31acb118-86b4-41eb-bf6f-3ef9697b47f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d73908ad335745ab842ca8e8b91924f7, user_id=e99835e9f0a24ba6a941814f820a61d6, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:03.873636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31acb118-86b4-41eb-bf6f-3ef9697b47f6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:03.973524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 908/1759] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:03 2022] PUT /identity/v3/projects/d73908ad335745ab842ca8e8b91924f7/users/e99835e9f0a24ba6a941814f820a61d6/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:03.977369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2747c1-0928-438b-af0b-77567a8012fa None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:03.977901 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2747c1-0928-438b-af0b-77567a8012fa None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:03.978138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2747c1-0928-438b-af0b-77567a8012fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:04.010904 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd513b5b97c9048029e9bb10799f246fe'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:04.012080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd513b5b97c9048029e9bb10799f246fe'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:04.012955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd513b5b97c9048029e9bb10799f246fe'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:04.056568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd513b5b97c9048029e9bb10799f246fe'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:04.075787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd513b5b97c9048029e9bb10799f246fe'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:04.141989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 852/1760] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:03 2022] DELETE /identity/v3/users/d513b5b97c9048029e9bb10799f246fe => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:04.179918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-df2747c1-0928-438b-af0b-77567a8012fa None None] MFA Rules not processed for user `e99835e9f0a24ba6a941814f820a61d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:04.182904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-df2747c1-0928-438b-af0b-77567a8012fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:04.422077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 909/1761] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:04.432845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:04.437745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:04.796494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21724415-8fe8-4949-955e-988d140d7534 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:04.809326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39b044-1263-4bf2-b706-66f26528e513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:04.810357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39b044-1263-4bf2-b706-66f26528e513 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:04.814640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39b044-1263-4bf2-b706-66f26528e513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:04.815267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39b044-1263-4bf2-b706-66f26528e513 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:04.818983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39b044-1263-4bf2-b706-66f26528e513 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:04.823320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39b044-1263-4bf2-b706-66f26528e513 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:04.883238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 853/1762] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:04 2022] POST /identity/v3/projects => generated 389 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:04.887432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2747c1-0928-438b-af0b-77567a8012fa None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:04.919979 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2747c1-0928-438b-af0b-77567a8012fa None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:04.934996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:05.229186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6af21-9f36-4006-b347-4eac096353f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:05.231108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6af21-9f36-4006-b347-4eac096353f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:05.231108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6af21-9f36-4006-b347-4eac096353f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:05.231108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6af21-9f36-4006-b347-4eac096353f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:05.232325 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6af21-9f36-4006-b347-4eac096353f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:05.235597 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a6af21-9f36-4006-b347-4eac096353f0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:05.332167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 910/1763] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:04 2022] POST /identity/v3/users => generated 301 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:05.334511 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39b044-1263-4bf2-b706-66f26528e513 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:05.370014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39b044-1263-4bf2-b706-66f26528e513 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:05.395734 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3383bb-37eb-484f-9d4e-6eab98544085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:05.396324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3383bb-37eb-484f-9d4e-6eab98544085 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:05.396324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3383bb-37eb-484f-9d4e-6eab98544085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:05.396663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3383bb-37eb-484f-9d4e-6eab98544085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:05.403447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3383bb-37eb-484f-9d4e-6eab98544085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:05.405152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3383bb-37eb-484f-9d4e-6eab98544085 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:05.457692 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6af21-9f36-4006-b347-4eac096353f0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:05.469195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 854/1764] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:05.469728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a6af21-9f36-4006-b347-4eac096353f0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:05.507574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc643ea-283b-49ea-b56f-f0c0a03f13fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:05.509223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc643ea-283b-49ea-b56f-f0c0a03f13fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:05.511387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc643ea-283b-49ea-b56f-f0c0a03f13fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:05.515659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc643ea-283b-49ea-b56f-f0c0a03f13fb admin admin] PATH_INFO: `/v3/projects/842bf7b15c2a43a6850317119ecb0cfb/users/c8d79f897d74451ca01914a605d5dc09/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:05.633830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc643ea-283b-49ea-b56f-f0c0a03f13fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=842bf7b15c2a43a6850317119ecb0cfb, user_id=c8d79f897d74451ca01914a605d5dc09, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:05.637579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc643ea-283b-49ea-b56f-f0c0a03f13fb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:05.706370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 911/1765] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:05 2022] PUT /identity/v3/projects/842bf7b15c2a43a6850317119ecb0cfb/users/c8d79f897d74451ca01914a605d5dc09/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:05.710861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3383bb-37eb-484f-9d4e-6eab98544085 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:05.765929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3383bb-37eb-484f-9d4e-6eab98544085 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:05.781730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:06.069390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25946904-83ce-402c-bc1a-bb958e319a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:06.071231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25946904-83ce-402c-bc1a-bb958e319a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:06.071231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25946904-83ce-402c-bc1a-bb958e319a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:06.071893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25946904-83ce-402c-bc1a-bb958e319a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:06.075642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25946904-83ce-402c-bc1a-bb958e319a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:06.084548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25946904-83ce-402c-bc1a-bb958e319a94 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:06.145976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 855/1766] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:06.146943 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc643ea-283b-49ea-b56f-f0c0a03f13fb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:06.211270 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc643ea-283b-49ea-b56f-f0c0a03f13fb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:06.255865 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb63405-aa25-4a99-a2be-e13b679d5e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:06.259126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb63405-aa25-4a99-a2be-e13b679d5e9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:06.260529 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb63405-aa25-4a99-a2be-e13b679d5e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:06.262486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb63405-aa25-4a99-a2be-e13b679d5e9d admin admin] PATH_INFO: `/v3/projects/842bf7b15c2a43a6850317119ecb0cfb/users/c8d79f897d74451ca01914a605d5dc09/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:06.277982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb63405-aa25-4a99-a2be-e13b679d5e9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=842bf7b15c2a43a6850317119ecb0cfb, user_id=c8d79f897d74451ca01914a605d5dc09, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:06.300867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb63405-aa25-4a99-a2be-e13b679d5e9d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:06.395733 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 912/1767] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:06 2022] PUT /identity/v3/projects/842bf7b15c2a43a6850317119ecb0cfb/users/c8d79f897d74451ca01914a605d5dc09/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:06.411018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b07d78-c3b8-45ae-b95f-682613240885 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:06.411312 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b07d78-c3b8-45ae-b95f-682613240885 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:06.411605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b07d78-c3b8-45ae-b95f-682613240885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:06.498322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-56b07d78-c3b8-45ae-b95f-682613240885 None None] MFA Rules not processed for user `c8d79f897d74451ca01914a605d5dc09`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:06.503621 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-56b07d78-c3b8-45ae-b95f-682613240885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:06.709067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab346dfd-d15e-46d1-9cb6-c7392a98bb32 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:06.709948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab346dfd-d15e-46d1-9cb6-c7392a98bb32 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:06.710548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab346dfd-d15e-46d1-9cb6-c7392a98bb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:06.810507 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ab346dfd-d15e-46d1-9cb6-c7392a98bb32 None None] MFA Rules not processed for user `5f095d3370be457cb6687ac8f8c9f8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:06.817092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ab346dfd-d15e-46d1-9cb6-c7392a98bb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:06.838680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 856/1768] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:06.844606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b07d78-c3b8-45ae-b95f-682613240885 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:06.849447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-56b07d78-c3b8-45ae-b95f-682613240885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:07.086868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 913/1769] 10.222.0.44 () {58 vars in 1015 bytes} [Thu Jun 23 11:55:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:07.116036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab346dfd-d15e-46d1-9cb6-c7392a98bb32 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:07.118078 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ab346dfd-d15e-46d1-9cb6-c7392a98bb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:07.146061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab346dfd-d15e-46d1-9cb6-c7392a98bb32 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:07.172042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bd54a-c574-4a4f-838f-4db879a38705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:07.174590 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bd54a-c574-4a4f-838f-4db879a38705 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:07.178118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bd54a-c574-4a4f-838f-4db879a38705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:07.188274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574bd54a-c574-4a4f-838f-4db879a38705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:07.216264 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bd54a-c574-4a4f-838f-4db879a38705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:07.222432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574bd54a-c574-4a4f-838f-4db879a38705 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:07.260581 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b07d78-c3b8-45ae-b95f-682613240885 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:07.261152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 914/1770] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:07.280015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3b450-98ec-4b61-b960-1131ae8df652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:07.285130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3b450-98ec-4b61-b960-1131ae8df652 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:07.285742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3b450-98ec-4b61-b960-1131ae8df652 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:07.288993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3b450-98ec-4b61-b960-1131ae8df652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:07.346684 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3b450-98ec-4b61-b960-1131ae8df652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:07.358927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3b450-98ec-4b61-b960-1131ae8df652 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:07.385180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 857/1771] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:07.831287 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bd54a-c574-4a4f-838f-4db879a38705 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:08.157630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:08.186504 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574bd54a-c574-4a4f-838f-4db879a38705 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:08.200077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:08.205031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:08.205685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:08.206167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] PATH_INFO: `/v3/users/e99835e9f0a24ba6a941814f820a61d6` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:08.207974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e99835e9f0a24ba6a941814f820a61d6)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:08.214611 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:08.351835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e99835e9f0a24ba6a941814f820a61d6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:08.352948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e99835e9f0a24ba6a941814f820a61d6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:08.354821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e99835e9f0a24ba6a941814f820a61d6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:08.386053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e99835e9f0a24ba6a941814f820a61d6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:08.398572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e99835e9f0a24ba6a941814f820a61d6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:08.463238 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 915/1772] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:07 2022] DELETE /identity/v3/users/e99835e9f0a24ba6a941814f820a61d6 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:08.465300 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0e962-099a-4fd7-9bf7-7422ddba1ee2 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:08.465818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0e962-099a-4fd7-9bf7-7422ddba1ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:08.466076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0e962-099a-4fd7-9bf7-7422ddba1ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:08.564223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b8b0e962-099a-4fd7-9bf7-7422ddba1ee2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:08.568466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b8b0e962-099a-4fd7-9bf7-7422ddba1ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:08.688145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:08.846794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 858/1773] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:08.880389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:08.883975 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:08.906201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad5971a-0ee1-44d9-bfc6-f77b64b0ab81 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:08.920878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a87fd-843a-47f5-a982-d856704d1cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:08.922211 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a87fd-843a-47f5-a982-d856704d1cd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:08.922673 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a87fd-843a-47f5-a982-d856704d1cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:08.923137 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a87fd-843a-47f5-a982-d856704d1cd5 admin admin] PATH_INFO: `/v3/projects/4c2cfa97ca86470c86f0d436da74f89d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:08.924838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a87fd-843a-47f5-a982-d856704d1cd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c2cfa97ca86470c86f0d436da74f89d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:08.929843 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a87fd-843a-47f5-a982-d856704d1cd5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:09.086047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8e0a87fd-843a-47f5-a982-d856704d1cd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c2cfa97ca86470c86f0d436da74f89d', 'request_id': 'req-8e0a87fd-843a-47f5-a982-d856704d1cd5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:09.102621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8e0a87fd-843a-47f5-a982-d856704d1cd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c2cfa97ca86470c86f0d436da74f89d', 'request_id': 'req-8e0a87fd-843a-47f5-a982-d856704d1cd5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:09.116141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8e0a87fd-843a-47f5-a982-d856704d1cd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c2cfa97ca86470c86f0d436da74f89d', 'request_id': 'req-8e0a87fd-843a-47f5-a982-d856704d1cd5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:09.144014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8e0a87fd-843a-47f5-a982-d856704d1cd5 admin admin] The token cache is being invalidate because project 4c2cfa97ca86470c86f0d436da74f89d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:09.144827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8e0a87fd-843a-47f5-a982-d856704d1cd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:09.147454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8e0a87fd-843a-47f5-a982-d856704d1cd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:09.159079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac3744-3af2-43fe-8830-7633d4fb4905 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:09.159470 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac3744-3af2-43fe-8830-7633d4fb4905 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:09.159654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac3744-3af2-43fe-8830-7633d4fb4905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:09.168809 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 916/1774] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:08 2022] DELETE /identity/v3/projects/4c2cfa97ca86470c86f0d436da74f89d => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:09.221173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-18ac3744-3af2-43fe-8830-7633d4fb4905 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:09.221173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-18ac3744-3af2-43fe-8830-7633d4fb4905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:09.451924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 859/1775] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:09.459147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e61a4c-8ab1-4565-877c-cbb10f2d56dc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:09.460547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e61a4c-8ab1-4565-877c-cbb10f2d56dc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:09.460547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e61a4c-8ab1-4565-877c-cbb10f2d56dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:09.497212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-49e61a4c-8ab1-4565-877c-cbb10f2d56dc None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:09.498484 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-49e61a4c-8ab1-4565-877c-cbb10f2d56dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:09.545796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 917/1776] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:09.547994 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac3744-3af2-43fe-8830-7633d4fb4905 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:09.571917 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac3744-3af2-43fe-8830-7633d4fb4905 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:09.603009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c420d87-3f2e-4180-9260-fbffa95d8bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:09.603717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c420d87-3f2e-4180-9260-fbffa95d8bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:09.609650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c420d87-3f2e-4180-9260-fbffa95d8bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:09.610621 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c420d87-3f2e-4180-9260-fbffa95d8bf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:09.617421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c420d87-3f2e-4180-9260-fbffa95d8bf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:09.617807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c420d87-3f2e-4180-9260-fbffa95d8bf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:09.625064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c420d87-3f2e-4180-9260-fbffa95d8bf9 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:09.699022 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 860/1777] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:09.702298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e61a4c-8ab1-4565-877c-cbb10f2d56dc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:09.723532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e61a4c-8ab1-4565-877c-cbb10f2d56dc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:09.749501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0f991-3c8d-4858-b19d-f1d87b54d6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:09.770047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c420d87-3f2e-4180-9260-fbffa95d8bf9 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:09.770410 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0f991-3c8d-4858-b19d-f1d87b54d6ea admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:09.771867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0c420d87-3f2e-4180-9260-fbffa95d8bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:09.772638 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0f991-3c8d-4858-b19d-f1d87b54d6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:09.775312 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0f991-3c8d-4858-b19d-f1d87b54d6ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:09.779342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0f991-3c8d-4858-b19d-f1d87b54d6ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:09.795154 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0f991-3c8d-4858-b19d-f1d87b54d6ea admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:09.994101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 918/1778] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:09 2022] POST /identity/v3/projects => generated 365 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:09.997730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0f991-3c8d-4858-b19d-f1d87b54d6ea admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:10.071076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c420d87-3f2e-4180-9260-fbffa95d8bf9 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:10.082016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:10.182464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0f991-3c8d-4858-b19d-f1d87b54d6ea admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:10.198860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c115c-89d0-49dd-9bbb-8582c89ab92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:10.203083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c115c-89d0-49dd-9bbb-8582c89ab92b admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:10.203083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c115c-89d0-49dd-9bbb-8582c89ab92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:10.203310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c115c-89d0-49dd-9bbb-8582c89ab92b admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:10.205607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c115c-89d0-49dd-9bbb-8582c89ab92b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:10.213228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c115c-89d0-49dd-9bbb-8582c89ab92b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:10.280651 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 919/1779] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:09 2022] POST /identity/v3/users => generated 283 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:10.282714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c115c-89d0-49dd-9bbb-8582c89ab92b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:10.316934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c115c-89d0-49dd-9bbb-8582c89ab92b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:10.350229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31955a-75db-42e6-a410-5eba9782fb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:10.356798 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31955a-75db-42e6-a410-5eba9782fb37 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:10.357155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31955a-75db-42e6-a410-5eba9782fb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:10.357446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31955a-75db-42e6-a410-5eba9782fb37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:10.363228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31955a-75db-42e6-a410-5eba9782fb37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:10.391127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31955a-75db-42e6-a410-5eba9782fb37 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:10.400570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aba529-9a2e-4b1f-b4f9-3b8b42d5d670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:10.401334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aba529-9a2e-4b1f-b4f9-3b8b42d5d670 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:10.401625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aba529-9a2e-4b1f-b4f9-3b8b42d5d670 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:10.401909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aba529-9a2e-4b1f-b4f9-3b8b42d5d670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:10.421908 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-39aba529-9a2e-4b1f-b4f9-3b8b42d5d670 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:10.446010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 920/1780] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:10.448735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31955a-75db-42e6-a410-5eba9782fb37 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:10.494878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31955a-75db-42e6-a410-5eba9782fb37 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:10.499623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aba529-9a2e-4b1f-b4f9-3b8b42d5d670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:10.501147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aba529-9a2e-4b1f-b4f9-3b8b42d5d670 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:10.515611 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b376c-3774-4475-ab15-7119518b84e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:10.519748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b376c-3774-4475-ab15-7119518b84e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:10.520289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b376c-3774-4475-ab15-7119518b84e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:10.520878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b376c-3774-4475-ab15-7119518b84e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:10.521864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b376c-3774-4475-ab15-7119518b84e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:10.530250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b376c-3774-4475-ab15-7119518b84e8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:10.580413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 921/1782] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:10.584085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b376c-3774-4475-ab15-7119518b84e8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:10.584758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 861/1782] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 808 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:10.639778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b376c-3774-4475-ab15-7119518b84e8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:10.673013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9913de-436d-4cc2-bde8-d075ecd250f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:10.675606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9913de-436d-4cc2-bde8-d075ecd250f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:10.675895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9913de-436d-4cc2-bde8-d075ecd250f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:10.676178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9913de-436d-4cc2-bde8-d075ecd250f6 admin admin] PATH_INFO: `/v3/projects/805b95c31b044d4ebd817e9a514de289/users/6af98c0b33514f81a88dcb73433141ce/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:10.777641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9913de-436d-4cc2-bde8-d075ecd250f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=805b95c31b044d4ebd817e9a514de289, user_id=6af98c0b33514f81a88dcb73433141ce, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:10.800052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9913de-436d-4cc2-bde8-d075ecd250f6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:10.869772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 922/1783] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:10 2022] PUT /identity/v3/projects/805b95c31b044d4ebd817e9a514de289/users/6af98c0b33514f81a88dcb73433141ce/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:10.869983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b6053-1049-4535-9039-0e219ce4ac14 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:10.870244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b6053-1049-4535-9039-0e219ce4ac14 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:10.870542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b6053-1049-4535-9039-0e219ce4ac14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:11.015041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1b7b6053-1049-4535-9039-0e219ce4ac14 None None] MFA Rules not processed for user `6af98c0b33514f81a88dcb73433141ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:11.018159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1b7b6053-1049-4535-9039-0e219ce4ac14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:11.069980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:11.337866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 862/1784] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:11.342500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9913de-436d-4cc2-bde8-d075ecd250f6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:11.604145 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9913de-436d-4cc2-bde8-d075ecd250f6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:11.618844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e61536-f946-4006-93ab-bbc721ea95f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:11.619688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e61536-f946-4006-93ab-bbc721ea95f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:11.620084 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e61536-f946-4006-93ab-bbc721ea95f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:11.620474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e61536-f946-4006-93ab-bbc721ea95f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:11.621394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e61536-f946-4006-93ab-bbc721ea95f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:11.623791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e61536-f946-4006-93ab-bbc721ea95f2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:11.699908 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b6053-1049-4535-9039-0e219ce4ac14 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:11.708777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 923/1785] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:11 2022] POST /identity/v3/projects => generated 363 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:11.724851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b6053-1049-4535-9039-0e219ce4ac14 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:11.920488 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7337472-49cb-451d-a8fd-6bd4560bac68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:11.921613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7337472-49cb-451d-a8fd-6bd4560bac68 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:11.922110 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7337472-49cb-451d-a8fd-6bd4560bac68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:11.922609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7337472-49cb-451d-a8fd-6bd4560bac68 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:11.925318 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7337472-49cb-451d-a8fd-6bd4560bac68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:11.935671 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7337472-49cb-451d-a8fd-6bd4560bac68 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:11.993441 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e61536-f946-4006-93ab-bbc721ea95f2 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:12.001599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 863/1786] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:11 2022] POST /identity/v3/users => generated 288 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:12.008250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e61536-f946-4006-93ab-bbc721ea95f2 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:12.021683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea82643-b218-40f6-b00f-9a2c149c1805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:12.028229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82643-b218-40f6-b00f-9a2c149c1805 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:12.028773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82643-b218-40f6-b00f-9a2c149c1805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:12.029161 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea82643-b218-40f6-b00f-9a2c149c1805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:12.030072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea82643-b218-40f6-b00f-9a2c149c1805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:12.031844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea82643-b218-40f6-b00f-9a2c149c1805 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:12.066744 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7337472-49cb-451d-a8fd-6bd4560bac68 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:12.081951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 924/1787] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:12.090060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7337472-49cb-451d-a8fd-6bd4560bac68 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:12.145811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63259a4a-7a17-4633-ac45-e22f9bae37ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:12.146722 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63259a4a-7a17-4633-ac45-e22f9bae37ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:12.148265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63259a4a-7a17-4633-ac45-e22f9bae37ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:12.150373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63259a4a-7a17-4633-ac45-e22f9bae37ef admin admin] PATH_INFO: `/v3/projects/6ef267d9d488405caaa1cc8f08815f23/users/0100cd5cbaf946a2b60d50e701b03f26/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:12.198297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63259a4a-7a17-4633-ac45-e22f9bae37ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ef267d9d488405caaa1cc8f08815f23, user_id=0100cd5cbaf946a2b60d50e701b03f26, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:12.204375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63259a4a-7a17-4633-ac45-e22f9bae37ef admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:12.259062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea82643-b218-40f6-b00f-9a2c149c1805 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:12.259591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 864/1788] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:12 2022] PUT /identity/v3/projects/6ef267d9d488405caaa1cc8f08815f23/users/0100cd5cbaf946a2b60d50e701b03f26/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:12.272255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea82643-b218-40f6-b00f-9a2c149c1805 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:12.404564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:12.434054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27222b94-2f2f-4d73-b96d-9b153c6b01f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:12.435246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222b94-2f2f-4d73-b96d-9b153c6b01f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:12.435765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222b94-2f2f-4d73-b96d-9b153c6b01f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:12.436273 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27222b94-2f2f-4d73-b96d-9b153c6b01f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:12.437285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222b94-2f2f-4d73-b96d-9b153c6b01f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:12.439240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27222b94-2f2f-4d73-b96d-9b153c6b01f5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:12.476321 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 925/1789] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:12.478211 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63259a4a-7a17-4633-ac45-e22f9bae37ef admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:12.495036 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63259a4a-7a17-4633-ac45-e22f9bae37ef admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:12.523683 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9c278-ce23-4482-9443-129c1370e4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:12.526357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9c278-ce23-4482-9443-129c1370e4f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:12.528230 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9c278-ce23-4482-9443-129c1370e4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:12.529914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9c278-ce23-4482-9443-129c1370e4f1 admin admin] PATH_INFO: `/v3/projects/6ef267d9d488405caaa1cc8f08815f23/users/0100cd5cbaf946a2b60d50e701b03f26/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:12.535830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9c278-ce23-4482-9443-129c1370e4f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ef267d9d488405caaa1cc8f08815f23, user_id=0100cd5cbaf946a2b60d50e701b03f26, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:12.543400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9c278-ce23-4482-9443-129c1370e4f1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:12.598441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 865/1790] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:12 2022] PUT /identity/v3/projects/6ef267d9d488405caaa1cc8f08815f23/users/0100cd5cbaf946a2b60d50e701b03f26/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:12.610953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1627884-8024-45f6-bfec-9bdbbe16adeb None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:12.611846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1627884-8024-45f6-bfec-9bdbbe16adeb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:12.612299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1627884-8024-45f6-bfec-9bdbbe16adeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:12.684301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c1627884-8024-45f6-bfec-9bdbbe16adeb None None] MFA Rules not processed for user `0100cd5cbaf946a2b60d50e701b03f26`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:12.687016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c1627884-8024-45f6-bfec-9bdbbe16adeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:12.827915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950ef8a-552c-4408-b4df-9c0a43c99004 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:12.829546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950ef8a-552c-4408-b4df-9c0a43c99004 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:12.830033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950ef8a-552c-4408-b4df-9c0a43c99004 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:12.832380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 866/1791] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 11:55:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 11:55:12.851162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f3521-3ddc-4a89-a64a-ad57156d4e2c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:12.851558 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f3521-3ddc-4a89-a64a-ad57156d4e2c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:12.851916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f3521-3ddc-4a89-a64a-ad57156d4e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:12.937221 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-5c5f3521-3ddc-4a89-a64a-ad57156d4e2c None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:12.946170 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5c5f3521-3ddc-4a89-a64a-ad57156d4e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:13.050138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 926/1792] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:12 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:13.065494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1627884-8024-45f6-bfec-9bdbbe16adeb None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:13.066932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c1627884-8024-45f6-bfec-9bdbbe16adeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:13.204731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:13.263121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 867/1793] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 11:55:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:13.299287 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1627884-8024-45f6-bfec-9bdbbe16adeb None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:13.323604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1deec67-c307-4b56-a4b0-ec14630967bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:13.324297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1deec67-c307-4b56-a4b0-ec14630967bd service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:13.324713 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1deec67-c307-4b56-a4b0-ec14630967bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:13.325003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1deec67-c307-4b56-a4b0-ec14630967bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:13.382553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1deec67-c307-4b56-a4b0-ec14630967bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:13.386499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1deec67-c307-4b56-a4b0-ec14630967bd service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:13.419227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f3521-3ddc-4a89-a64a-ad57156d4e2c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:13.422031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5c5f3521-3ddc-4a89-a64a-ad57156d4e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:13.434676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 927/1794] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:13 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:13.435613 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1deec67-c307-4b56-a4b0-ec14630967bd service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:13.444267 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e1deec67-c307-4b56-a4b0-ec14630967bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:13.478904 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5f3521-3ddc-4a89-a64a-ad57156d4e2c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:13.523470 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e49ca1-209a-4e8c-ac5d-90d5f29e0dc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:13.533081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e49ca1-209a-4e8c-ac5d-90d5f29e0dc0 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:13.533806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e49ca1-209a-4e8c-ac5d-90d5f29e0dc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:13.533806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e49ca1-209a-4e8c-ac5d-90d5f29e0dc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:13.582918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e49ca1-209a-4e8c-ac5d-90d5f29e0dc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:13.592469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e49ca1-209a-4e8c-ac5d-90d5f29e0dc0 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:13.677231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 868/1795] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:13.678402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e49ca1-209a-4e8c-ac5d-90d5f29e0dc0 service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:13.704187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e49ca1-209a-4e8c-ac5d-90d5f29e0dc0 service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:13.715064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfa2c5-2f67-41e7-b284-340461161d4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:13.715971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfa2c5-2f67-41e7-b284-340461161d4b service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:13.716354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfa2c5-2f67-41e7-b284-340461161d4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:13.716648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfa2c5-2f67-41e7-b284-340461161d4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:13.825443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1deec67-c307-4b56-a4b0-ec14630967bd service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:13.838847 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:13.839786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:13.840092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:13.842005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] PATH_INFO: `/v3/users/4730a01a041e4f13b972877f835e9f02` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:13.844389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4730a01a041e4f13b972877f835e9f02)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:13.846441 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:13.886048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:13.971043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4730a01a041e4f13b972877f835e9f02'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:13.972188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfa2c5-2f67-41e7-b284-340461161d4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:13.973175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4730a01a041e4f13b972877f835e9f02'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:13.975369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4730a01a041e4f13b972877f835e9f02'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:13.978004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfa2c5-2f67-41e7-b284-340461161d4b service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:14.006385 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 869/1796] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:14.024900 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4730a01a041e4f13b972877f835e9f02'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:14.044118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4730a01a041e4f13b972877f835e9f02'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:14.098375 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 928/1797] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:13 2022] DELETE /identity/v3/users/4730a01a041e4f13b972877f835e9f02 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:14.100121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a7ce2-0c02-4b8c-9514-a5a10599ac15 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:14.100452 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a7ce2-0c02-4b8c-9514-a5a10599ac15 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:14.100843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a7ce2-0c02-4b8c-9514-a5a10599ac15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:14.144906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1f2a7ce2-0c02-4b8c-9514-a5a10599ac15 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:14.153261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1f2a7ce2-0c02-4b8c-9514-a5a10599ac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:14.475421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 870/1798] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:14.559367 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:14.561393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:14.734021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99d649-0b48-48cd-b7ea-bd8043fd973d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:14.745238 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0f883-9708-4433-86d2-8fa5c677db73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:14.745926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0f883-9708-4433-86d2-8fa5c677db73 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:14.746215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0f883-9708-4433-86d2-8fa5c677db73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:14.746497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0f883-9708-4433-86d2-8fa5c677db73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:14.777810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0f883-9708-4433-86d2-8fa5c677db73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:14.778676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0f883-9708-4433-86d2-8fa5c677db73 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:14.824748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 929/1799] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:15.800875 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a7ce2-0c02-4b8c-9514-a5a10599ac15 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:15.802551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1f2a7ce2-0c02-4b8c-9514-a5a10599ac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:15.835714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a7ce2-0c02-4b8c-9514-a5a10599ac15 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:15.859877 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:15.860962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:15.861319 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:15.861636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] PATH_INFO: `/v3/projects/d73908ad335745ab842ca8e8b91924f7` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:15.869989 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d73908ad335745ab842ca8e8b91924f7)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:15.872088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:15.988265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd73908ad335745ab842ca8e8b91924f7', 'request_id': 'req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:16.009762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd73908ad335745ab842ca8e8b91924f7', 'request_id': 'req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:16.021953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd73908ad335745ab842ca8e8b91924f7', 'request_id': 'req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:16.053179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] The token cache is being invalidate because project d73908ad335745ab842ca8e8b91924f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:16.053179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:16.061958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:16.066548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 871/1800] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:15 2022] DELETE /identity/v3/projects/d73908ad335745ab842ca8e8b91924f7 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:16.068292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0f883-9708-4433-86d2-8fa5c677db73 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:16.085466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0f883-9708-4433-86d2-8fa5c677db73 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:16.317065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:16.317065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:16.317065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:16.317065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] PATH_INFO: `/v3/users/c8d79f897d74451ca01914a605d5dc09` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:16.317485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d79f897d74451ca01914a605d5dc09)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:16.317485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:16.420220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:16.490744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d79f897d74451ca01914a605d5dc09'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:16.496894 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d79f897d74451ca01914a605d5dc09'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:16.498083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d79f897d74451ca01914a605d5dc09'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:16.552551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d79f897d74451ca01914a605d5dc09'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:16.580390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d79f897d74451ca01914a605d5dc09'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:16.609450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 930/1801] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:16 2022] DELETE /identity/v3/users/c8d79f897d74451ca01914a605d5dc09 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:17.109548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:17.115034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:17.139360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:17.159743 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22760f0-2f0e-4cc1-8edf-54cb70151e6d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:17.243290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:17.245704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:17.246060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:17.246449 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] PATH_INFO: `/v3/projects/efb0562b0b4349d190b07dce0d1a96dd` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:17.248138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb0562b0b4349d190b07dce0d1a96dd)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:17.251880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:17.339604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb0562b0b4349d190b07dce0d1a96dd', 'request_id': 'req-94136283-9385-408f-9723-8534ea46d3bc'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.345322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:17.347060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:17.349903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb0562b0b4349d190b07dce0d1a96dd', 'request_id': 'req-94136283-9385-408f-9723-8534ea46d3bc'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.358720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb0562b0b4349d190b07dce0d1a96dd', 'request_id': 'req-94136283-9385-408f-9723-8534ea46d3bc'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.370477 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d5035-f9d1-49e3-9125-acd31a9ee280 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:17.377666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] The token cache is being invalidate because project efb0562b0b4349d190b07dce0d1a96dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:17.378197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.379157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.383782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 872/1802] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:17 2022] DELETE /identity/v3/projects/efb0562b0b4349d190b07dce0d1a96dd => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:17.385109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:17.386986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:17.449583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38751dd7-c697-43ca-bf34-09e822f4d8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:17.450961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38751dd7-c697-43ca-bf34-09e822f4d8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:17.451425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38751dd7-c697-43ca-bf34-09e822f4d8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:17.452018 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38751dd7-c697-43ca-bf34-09e822f4d8ef admin admin] PATH_INFO: `/v3/projects/842bf7b15c2a43a6850317119ecb0cfb` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:17.454966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38751dd7-c697-43ca-bf34-09e822f4d8ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=842bf7b15c2a43a6850317119ecb0cfb)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:17.458064 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38751dd7-c697-43ca-bf34-09e822f4d8ef admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:17.518705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94136283-9385-408f-9723-8534ea46d3bc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:17.553453 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-38751dd7-c697-43ca-bf34-09e822f4d8ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '842bf7b15c2a43a6850317119ecb0cfb', 'request_id': 'req-38751dd7-c697-43ca-bf34-09e822f4d8ef'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.571496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-38751dd7-c697-43ca-bf34-09e822f4d8ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '842bf7b15c2a43a6850317119ecb0cfb', 'request_id': 'req-38751dd7-c697-43ca-bf34-09e822f4d8ef'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.581715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-38751dd7-c697-43ca-bf34-09e822f4d8ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '842bf7b15c2a43a6850317119ecb0cfb', 'request_id': 'req-38751dd7-c697-43ca-bf34-09e822f4d8ef'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.604664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-38751dd7-c697-43ca-bf34-09e822f4d8ef admin admin] The token cache is being invalidate because project 842bf7b15c2a43a6850317119ecb0cfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:17.605495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-38751dd7-c697-43ca-bf34-09e822f4d8ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.606219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-38751dd7-c697-43ca-bf34-09e822f4d8ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.613374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 931/1803] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:17 2022] DELETE /identity/v3/projects/842bf7b15c2a43a6850317119ecb0cfb => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:17.615353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478955f-eba7-4074-b74a-4362615fdd06 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:17.615784 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478955f-eba7-4074-b74a-4362615fdd06 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:17.616170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478955f-eba7-4074-b74a-4362615fdd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:17.626361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:17.671755 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca867c-6bf7-47e3-8522-05f4ee9ff1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:17.672898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca867c-6bf7-47e3-8522-05f4ee9ff1bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:17.673398 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca867c-6bf7-47e3-8522-05f4ee9ff1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:17.673815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca867c-6bf7-47e3-8522-05f4ee9ff1bc admin admin] PATH_INFO: `/v3/users/d4082afd3af942c1aa17985a7b665e6c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:17.675233 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca867c-6bf7-47e3-8522-05f4ee9ff1bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4082afd3af942c1aa17985a7b665e6c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:17.677715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca867c-6bf7-47e3-8522-05f4ee9ff1bc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:17.690966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3478955f-eba7-4074-b74a-4362615fdd06 None None] MFA Rules not processed for user `9cc891cc8172490394d2bfa3c8d1dcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:17.693585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3478955f-eba7-4074-b74a-4362615fdd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:17.759953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:17.788000 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 932/1804] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:17.789729 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb198ca-300f-4532-ba70-2252c1bf875b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:17.790148 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb198ca-300f-4532-ba70-2252c1bf875b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:17.790841 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb198ca-300f-4532-ba70-2252c1bf875b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:17.817090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-88ca867c-6bf7-47e3-8522-05f4ee9ff1bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4082afd3af942c1aa17985a7b665e6c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.818460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-88ca867c-6bf7-47e3-8522-05f4ee9ff1bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4082afd3af942c1aa17985a7b665e6c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.820198 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-88ca867c-6bf7-47e3-8522-05f4ee9ff1bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4082afd3af942c1aa17985a7b665e6c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.838114 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9bb198ca-300f-4532-ba70-2252c1bf875b None None] MFA Rules not processed for user `ebfb5a6c657a4fd592ff9e67348627eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:17.842109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9bb198ca-300f-4532-ba70-2252c1bf875b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:17.893814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-88ca867c-6bf7-47e3-8522-05f4ee9ff1bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4082afd3af942c1aa17985a7b665e6c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.923236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-88ca867c-6bf7-47e3-8522-05f4ee9ff1bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4082afd3af942c1aa17985a7b665e6c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:17.955986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 873/1805] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:17 2022] DELETE /identity/v3/users/d4082afd3af942c1aa17985a7b665e6c => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:17.957826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df4286-7613-4689-8b1e-fb70e70f2f74 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:17.958295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df4286-7613-4689-8b1e-fb70e70f2f74 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:17.958721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67df4286-7613-4689-8b1e-fb70e70f2f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:17.970389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 933/1806] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:17.970389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb198ca-300f-4532-ba70-2252c1bf875b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:17.985688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9bb198ca-300f-4532-ba70-2252c1bf875b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:18.005619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-67df4286-7613-4689-8b1e-fb70e70f2f74 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:18.008207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-67df4286-7613-4689-8b1e-fb70e70f2f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:18.052627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb198ca-300f-4532-ba70-2252c1bf875b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:18.172145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 874/1807] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:18.177742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:18.178550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.179460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.179745 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] PATH_INFO: `/v3/users/6af98c0b33514f81a88dcb73433141ce` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.181697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6af98c0b33514f81a88dcb73433141ce)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:18.182845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506abf5-5ad4-49e7-9f21-09ceb589dd7d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.183902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:18.185039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506abf5-5ad4-49e7-9f21-09ceb589dd7d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.185479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d506abf5-5ad4-49e7-9f21-09ceb589dd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.211560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d506abf5-5ad4-49e7-9f21-09ceb589dd7d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:18.214032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d506abf5-5ad4-49e7-9f21-09ceb589dd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:18.224155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 875/1808] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:18.229218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506abf5-5ad4-49e7-9f21-09ceb589dd7d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:18.253823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d506abf5-5ad4-49e7-9f21-09ceb589dd7d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:18.260698 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6af98c0b33514f81a88dcb73433141ce'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:18.261398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6af98c0b33514f81a88dcb73433141ce'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:18.262100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6af98c0b33514f81a88dcb73433141ce'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:18.274591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c79cbe-ad86-49f1-9031-39f11ed1201b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:18.284779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c79cbe-ad86-49f1-9031-39f11ed1201b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.285090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c79cbe-ad86-49f1-9031-39f11ed1201b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.285372 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c79cbe-ad86-49f1-9031-39f11ed1201b admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.286139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c79cbe-ad86-49f1-9031-39f11ed1201b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:18.286454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c79cbe-ad86-49f1-9031-39f11ed1201b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:18.289356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c79cbe-ad86-49f1-9031-39f11ed1201b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:18.296409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6af98c0b33514f81a88dcb73433141ce'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:18.306447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 876/1809] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:18.307954 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd99d9-78b8-4a1c-b23a-6cb0b9bf1dfa None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.308252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd99d9-78b8-4a1c-b23a-6cb0b9bf1dfa None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.308541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd99d9-78b8-4a1c-b23a-6cb0b9bf1dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.314591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6af98c0b33514f81a88dcb73433141ce'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:18.327633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bffd99d9-78b8-4a1c-b23a-6cb0b9bf1dfa None None] MFA Rules not processed for user `9ab84989dd384d15b316b20cc65e792a`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:18.328945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bffd99d9-78b8-4a1c-b23a-6cb0b9bf1dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:18.342057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 934/1810] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:17 2022] DELETE /identity/v3/users/6af98c0b33514f81a88dcb73433141ce => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:18.343393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:18.349350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:18.382173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:18.393460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:18.434720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 877/1811] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:18.436185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85259888-aa0e-4c07-9913-ec1ec3297a9c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.436481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85259888-aa0e-4c07-9913-ec1ec3297a9c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.436799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85259888-aa0e-4c07-9913-ec1ec3297a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.481344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-85259888-aa0e-4c07-9913-ec1ec3297a9c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:18.482192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-85259888-aa0e-4c07-9913-ec1ec3297a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:18.501918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0484e-eadd-4f97-89bb-8d7787ffc6c5 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:18.504495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 878/1812] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:18.507004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d55e3-ab43-431c-bfdf-ce9675ece4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:18.507674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d55e3-ab43-431c-bfdf-ce9675ece4ba admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.507960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d55e3-ab43-431c-bfdf-ce9675ece4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.508247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d55e3-ab43-431c-bfdf-ce9675ece4ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.508988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d55e3-ab43-431c-bfdf-ce9675ece4ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:18.511099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d55e3-ab43-431c-bfdf-ce9675ece4ba admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:18.532262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85259888-aa0e-4c07-9913-ec1ec3297a9c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:18.536208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-85259888-aa0e-4c07-9913-ec1ec3297a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:18.560614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 935/1813] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:18 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:18.561568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d55e3-ab43-431c-bfdf-ce9675ece4ba admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:18.648289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85259888-aa0e-4c07-9913-ec1ec3297a9c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:18.665807 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d55e3-ab43-431c-bfdf-ce9675ece4ba admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:18.672688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e556b-ff5e-4945-8306-fd9798de564b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:18.673612 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e556b-ff5e-4945-8306-fd9798de564b admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.673767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e556b-ff5e-4945-8306-fd9798de564b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.674464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e556b-ff5e-4945-8306-fd9798de564b admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.678164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e556b-ff5e-4945-8306-fd9798de564b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:18.680858 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e556b-ff5e-4945-8306-fd9798de564b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:18.735596 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 936/1814] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:18 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:18.739867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e556b-ff5e-4945-8306-fd9798de564b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:18.759937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e556b-ff5e-4945-8306-fd9798de564b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:18.774697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc09e3e-f84c-4115-9f43-1211db2e6615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:18.775384 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc09e3e-f84c-4115-9f43-1211db2e6615 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.775679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc09e3e-f84c-4115-9f43-1211db2e6615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.775950 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc09e3e-f84c-4115-9f43-1211db2e6615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.776708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc09e3e-f84c-4115-9f43-1211db2e6615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:18.778748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc09e3e-f84c-4115-9f43-1211db2e6615 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:18.799274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85697f2-ad5d-476e-9e51-f70e1381d71f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:18.800100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85697f2-ad5d-476e-9e51-f70e1381d71f service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.800403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85697f2-ad5d-476e-9e51-f70e1381d71f service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.800697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85697f2-ad5d-476e-9e51-f70e1381d71f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.803811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 937/1815] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:18.807136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa51e21-12b8-4cee-b437-e122eacef66a None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.807136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa51e21-12b8-4cee-b437-e122eacef66a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.807136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa51e21-12b8-4cee-b437-e122eacef66a None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.811294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 938/1816] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 11:55:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:55:18.815690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa51e21-12b8-4cee-b437-e122eacef66a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:18.822662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85697f2-ad5d-476e-9e51-f70e1381d71f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:18.823588 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85697f2-ad5d-476e-9e51-f70e1381d71f service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:18.849886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 879/1817] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:18.850437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa51e21-12b8-4cee-b437-e122eacef66a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:18.856440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef8dee-8793-4775-8741-7a9f38355013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:18.857158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef8dee-8793-4775-8741-7a9f38355013 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.857445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef8dee-8793-4775-8741-7a9f38355013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.857722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef8dee-8793-4775-8741-7a9f38355013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.858455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef8dee-8793-4775-8741-7a9f38355013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:18.860090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef8dee-8793-4775-8741-7a9f38355013 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:18.870040 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c25cfc-cf0c-498c-a20c-6d05b043fdeb None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.870338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c25cfc-cf0c-498c-a20c-6d05b043fdeb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.870621 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c25cfc-cf0c-498c-a20c-6d05b043fdeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.893037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 939/1818] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:18.894032 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef8dee-8793-4775-8741-7a9f38355013 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:18.901785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-83c25cfc-cf0c-498c-a20c-6d05b043fdeb None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:18.906222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-83c25cfc-cf0c-498c-a20c-6d05b043fdeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:18.913895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef8dee-8793-4775-8741-7a9f38355013 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:18.935479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818290b-00de-4aea-88e8-5fc4c50475f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:18.937084 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818290b-00de-4aea-88e8-5fc4c50475f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.937712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818290b-00de-4aea-88e8-5fc4c50475f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.938212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818290b-00de-4aea-88e8-5fc4c50475f2 admin admin] PATH_INFO: `/v3/projects/ca4bf7461ca846e4871dc1a793dd0dc8/users/d15ad002cb914f14866e7608b67d305b/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:18.959096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c818290b-00de-4aea-88e8-5fc4c50475f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca4bf7461ca846e4871dc1a793dd0dc8, user_id=d15ad002cb914f14866e7608b67d305b, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:18.962869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c818290b-00de-4aea-88e8-5fc4c50475f2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:18.984237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 940/1819] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:18 2022] PUT /identity/v3/projects/ca4bf7461ca846e4871dc1a793dd0dc8/users/d15ad002cb914f14866e7608b67d305b/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:18.998002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20e672-523a-4d83-be0c-8337935b0fe7 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:18.998487 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20e672-523a-4d83-be0c-8337935b0fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:18.999754 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20e672-523a-4d83-be0c-8337935b0fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:19.047790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 880/1820] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 11:55:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:19.048970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8a20e672-523a-4d83-be0c-8337935b0fe7 None None] MFA Rules not processed for user `d15ad002cb914f14866e7608b67d305b`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:19.050312 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8a20e672-523a-4d83-be0c-8337935b0fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:19.111293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:19.164345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 941/1821] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:19.166289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c25cfc-cf0c-498c-a20c-6d05b043fdeb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:19.178043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c25cfc-cf0c-498c-a20c-6d05b043fdeb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:19.209765 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:19.258782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a1c1f-8d89-4583-bb0b-497dfac2411a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:19.259596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a1c1f-8d89-4583-bb0b-497dfac2411a admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:19.259958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a1c1f-8d89-4583-bb0b-497dfac2411a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:19.260312 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a1c1f-8d89-4583-bb0b-497dfac2411a admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:19.261289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a1c1f-8d89-4583-bb0b-497dfac2411a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:19.263676 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a1c1f-8d89-4583-bb0b-497dfac2411a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:19.296910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 881/1822] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:19 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:19.301224 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20e672-523a-4d83-be0c-8337935b0fe7 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:19.313110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20e672-523a-4d83-be0c-8337935b0fe7 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:19.389566 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdbd95-ca08-429d-af40-284c961bed70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:19.390474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdbd95-ca08-429d-af40-284c961bed70 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:19.390768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdbd95-ca08-429d-af40-284c961bed70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:19.391090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdbd95-ca08-429d-af40-284c961bed70 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:19.391808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdbd95-ca08-429d-af40-284c961bed70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:19.393812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fdbd95-ca08-429d-af40-284c961bed70 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:19.419432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 942/1823] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:19 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:19.423750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a1c1f-8d89-4583-bb0b-497dfac2411a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:19.434656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a1c1f-8d89-4583-bb0b-497dfac2411a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:19.441013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32c9eb-737f-42ba-bd09-22e5ddc437be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:19.442330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32c9eb-737f-42ba-bd09-22e5ddc437be admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:19.442330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32c9eb-737f-42ba-bd09-22e5ddc437be admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:19.442536 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32c9eb-737f-42ba-bd09-22e5ddc437be admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:19.443129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32c9eb-737f-42ba-bd09-22e5ddc437be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:19.444828 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af32c9eb-737f-42ba-bd09-22e5ddc437be admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:19.463365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 882/1824] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:19.465254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdbd95-ca08-429d-af40-284c961bed70 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:19.478920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fdbd95-ca08-429d-af40-284c961bed70 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:19.484698 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85aae6a-35ca-49cd-aae1-d30b3c10c02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:19.485648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85aae6a-35ca-49cd-aae1-d30b3c10c02d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:19.486117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85aae6a-35ca-49cd-aae1-d30b3c10c02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:19.486408 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85aae6a-35ca-49cd-aae1-d30b3c10c02d admin admin] PATH_INFO: `/v3/projects/418bba389a8249f8ad93cf79d589d968/users/68204bed43344af9ba7a83c6786589b8/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:19.510684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85aae6a-35ca-49cd-aae1-d30b3c10c02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=418bba389a8249f8ad93cf79d589d968, user_id=68204bed43344af9ba7a83c6786589b8, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:19.515006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85aae6a-35ca-49cd-aae1-d30b3c10c02d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:19.538007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 943/1825] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:19 2022] PUT /identity/v3/projects/418bba389a8249f8ad93cf79d589d968/users/68204bed43344af9ba7a83c6786589b8/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:19.543508 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32c9eb-737f-42ba-bd09-22e5ddc437be admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:19.556774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af32c9eb-737f-42ba-bd09-22e5ddc437be admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:19.645572 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60101f4d-5765-49a8-b33f-a84b780a9200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:19.646353 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60101f4d-5765-49a8-b33f-a84b780a9200 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:19.646655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60101f4d-5765-49a8-b33f-a84b780a9200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:19.646930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60101f4d-5765-49a8-b33f-a84b780a9200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:19.648386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60101f4d-5765-49a8-b33f-a84b780a9200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:19.651014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60101f4d-5765-49a8-b33f-a84b780a9200 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:19.667522 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 883/1826] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:19.671659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85aae6a-35ca-49cd-aae1-d30b3c10c02d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:19.683437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85aae6a-35ca-49cd-aae1-d30b3c10c02d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:19.689297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4875929-4576-4b95-8999-82fb4a0e7797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:19.690306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4875929-4576-4b95-8999-82fb4a0e7797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:19.690668 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4875929-4576-4b95-8999-82fb4a0e7797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:19.691128 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4875929-4576-4b95-8999-82fb4a0e7797 admin admin] PATH_INFO: `/v3/projects/418bba389a8249f8ad93cf79d589d968/users/68204bed43344af9ba7a83c6786589b8/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:19.693362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4875929-4576-4b95-8999-82fb4a0e7797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=418bba389a8249f8ad93cf79d589d968, user_id=68204bed43344af9ba7a83c6786589b8, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:19.696713 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4875929-4576-4b95-8999-82fb4a0e7797 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:19.708999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 944/1827] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:19 2022] PUT /identity/v3/projects/418bba389a8249f8ad93cf79d589d968/users/68204bed43344af9ba7a83c6786589b8/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:19.718816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4be96-0cfb-41dd-b615-5dd9056a802f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:19.719109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4be96-0cfb-41dd-b615-5dd9056a802f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:19.719393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4be96-0cfb-41dd-b615-5dd9056a802f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:19.753360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-04b4be96-0cfb-41dd-b615-5dd9056a802f None None] MFA Rules not processed for user `68204bed43344af9ba7a83c6786589b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:19.754637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-04b4be96-0cfb-41dd-b615-5dd9056a802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:19.888707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4875929-4576-4b95-8999-82fb4a0e7797 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:19.890535 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f4875929-4576-4b95-8999-82fb4a0e7797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:19.892029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 884/1828] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:19.906323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4be96-0cfb-41dd-b615-5dd9056a802f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:19.907764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-04b4be96-0cfb-41dd-b615-5dd9056a802f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:19.977261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:20.021560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4875929-4576-4b95-8999-82fb4a0e7797 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:20.029710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:20.030804 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:20.031089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:20.031371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] PATH_INFO: `/v3/projects/ffa0c5c6339c46cbbadcb29581cbbaa0` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:20.034396 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa0c5c6339c46cbbadcb29581cbbaa0)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:20.038942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:20.062633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4be96-0cfb-41dd-b615-5dd9056a802f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:20.074623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31613c4f-cf51-4f43-b767-44848847ffbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:20.074871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:20.075400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613c4f-cf51-4f43-b767-44848847ffbb service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:20.075594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613c4f-cf51-4f43-b767-44848847ffbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:20.075882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31613c4f-cf51-4f43-b767-44848847ffbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:20.095612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31613c4f-cf51-4f43-b767-44848847ffbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:20.097152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31613c4f-cf51-4f43-b767-44848847ffbb service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:20.121919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 885/1829] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:20.156795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa0c5c6339c46cbbadcb29581cbbaa0', 'request_id': 'req-538f79dd-10da-4af7-a1a4-a1601630fc64'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:20.174690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa0c5c6339c46cbbadcb29581cbbaa0', 'request_id': 'req-538f79dd-10da-4af7-a1a4-a1601630fc64'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:20.189458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa0c5c6339c46cbbadcb29581cbbaa0', 'request_id': 'req-538f79dd-10da-4af7-a1a4-a1601630fc64'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:20.212745 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] The token cache is being invalidate because project ffa0c5c6339c46cbbadcb29581cbbaa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:20.212972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:20.213949 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:20.217346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 945/1830] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:19 2022] DELETE /identity/v3/projects/ffa0c5c6339c46cbbadcb29581cbbaa0 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:20.222200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c077f-d6b0-42f0-9148-e0a0d0cc7982 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:20.222652 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c077f-d6b0-42f0-9148-e0a0d0cc7982 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:20.222740 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c077f-d6b0-42f0-9148-e0a0d0cc7982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:20.259509 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3a0c077f-d6b0-42f0-9148-e0a0d0cc7982 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:20.261242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3a0c077f-d6b0-42f0-9148-e0a0d0cc7982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:20.307305 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:20.314142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:20.441853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 886/1831] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:20.442799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c077f-d6b0-42f0-9148-e0a0d0cc7982 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:20.448123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3a0c077f-d6b0-42f0-9148-e0a0d0cc7982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:20.462105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f79dd-10da-4af7-a1a4-a1601630fc64 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:20.473523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f4183-f783-454e-bbae-51ba775b8e4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:20.476745 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f4183-f783-454e-bbae-51ba775b8e4e service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:20.476952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f4183-f783-454e-bbae-51ba775b8e4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:20.477236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f4183-f783-454e-bbae-51ba775b8e4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:20.483613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c077f-d6b0-42f0-9148-e0a0d0cc7982 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:20.484162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a22f4183-f783-454e-bbae-51ba775b8e4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:20.499447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:20.500391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:20.500612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:20.500919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] PATH_INFO: `/v3/projects/805b95c31b044d4ebd817e9a514de289` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:20.505395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] RBAC: Authorizing `identity:delete_project(project_id=805b95c31b044d4ebd817e9a514de289)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:20.507506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:20.633041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '805b95c31b044d4ebd817e9a514de289', 'request_id': 'req-75c48b4c-648d-4033-b9ec-4db32da6b664'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:20.653478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '805b95c31b044d4ebd817e9a514de289', 'request_id': 'req-75c48b4c-648d-4033-b9ec-4db32da6b664'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:20.671883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f4183-f783-454e-bbae-51ba775b8e4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:20.673723 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f4183-f783-454e-bbae-51ba775b8e4e service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:20.687057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 946/1832] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:20.688227 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f3930-78d9-4fbd-93f6-3c2f92b13c41 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:20.688521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f3930-78d9-4fbd-93f6-3c2f92b13c41 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:20.688841 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f3930-78d9-4fbd-93f6-3c2f92b13c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:20.694967 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '805b95c31b044d4ebd817e9a514de289', 'request_id': 'req-75c48b4c-648d-4033-b9ec-4db32da6b664'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:20.750233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1c1f3930-78d9-4fbd-93f6-3c2f92b13c41 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:20.751494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1c1f3930-78d9-4fbd-93f6-3c2f92b13c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:20.757196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] The token cache is being invalidate because project 805b95c31b044d4ebd817e9a514de289 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:20.757634 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:20.761966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:20.767631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 887/1833] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:20 2022] DELETE /identity/v3/projects/805b95c31b044d4ebd817e9a514de289 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:20.771397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:20.777797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:20.934866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 947/1834] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:20.936702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f3930-78d9-4fbd-93f6-3c2f92b13c41 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:20.939235 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1c1f3930-78d9-4fbd-93f6-3c2f92b13c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:20.953389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:20.999036 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:21.032023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c48b4c-648d-4033-b9ec-4db32da6b664 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:21.044155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:21.046438 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:21.046764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:21.047047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] PATH_INFO: `/v3/users/0100cd5cbaf946a2b60d50e701b03f26` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:21.048453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0100cd5cbaf946a2b60d50e701b03f26)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:21.057136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:21.113883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f3930-78d9-4fbd-93f6-3c2f92b13c41 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:21.132717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793505bb-8a06-4a3d-8abf-dd97e764f4de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:21.133693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793505bb-8a06-4a3d-8abf-dd97e764f4de service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:21.134222 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793505bb-8a06-4a3d-8abf-dd97e764f4de service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:21.134725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793505bb-8a06-4a3d-8abf-dd97e764f4de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:21.138170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-793505bb-8a06-4a3d-8abf-dd97e764f4de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:21.147138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0100cd5cbaf946a2b60d50e701b03f26'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:21.149283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0100cd5cbaf946a2b60d50e701b03f26'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:21.153759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0100cd5cbaf946a2b60d50e701b03f26'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:21.164111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793505bb-8a06-4a3d-8abf-dd97e764f4de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:21.168259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793505bb-8a06-4a3d-8abf-dd97e764f4de service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:21.177313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0100cd5cbaf946a2b60d50e701b03f26'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:21.191222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0100cd5cbaf946a2b60d50e701b03f26'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:21.193906 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 948/1835] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:21.195370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793505bb-8a06-4a3d-8abf-dd97e764f4de service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:21.218172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 888/1836] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:20 2022] DELETE /identity/v3/users/0100cd5cbaf946a2b60d50e701b03f26 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:21.219162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:21.222186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:21.238381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793505bb-8a06-4a3d-8abf-dd97e764f4de service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:21.359607 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cd5e2-d060-4a2b-a5a2-3131aad04943 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:21.377385 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ff2a3-981c-497b-96b7-d72fa70dfd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:21.378737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff2a3-981c-497b-96b7-d72fa70dfd2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:21.378737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff2a3-981c-497b-96b7-d72fa70dfd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:21.378941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ff2a3-981c-497b-96b7-d72fa70dfd2b admin admin] PATH_INFO: `/v3/users/d15ad002cb914f14866e7608b67d305b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:21.385520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ff2a3-981c-497b-96b7-d72fa70dfd2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d15ad002cb914f14866e7608b67d305b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:21.387694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ff2a3-981c-497b-96b7-d72fa70dfd2b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:21.458243 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd189850-4e4d-4af7-afd6-c31455f15d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:21.460966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd189850-4e4d-4af7-afd6-c31455f15d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:21.461855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd189850-4e4d-4af7-afd6-c31455f15d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:21.462956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd189850-4e4d-4af7-afd6-c31455f15d35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:21.465527 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd189850-4e4d-4af7-afd6-c31455f15d35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:21.466657 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd189850-4e4d-4af7-afd6-c31455f15d35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:21.472960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd189850-4e4d-4af7-afd6-c31455f15d35 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:21.492810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 949/1837] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:21.507607 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-637ff2a3-981c-497b-96b7-d72fa70dfd2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd15ad002cb914f14866e7608b67d305b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:21.507925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd189850-4e4d-4af7-afd6-c31455f15d35 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:21.509883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-637ff2a3-981c-497b-96b7-d72fa70dfd2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd15ad002cb914f14866e7608b67d305b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:21.510548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-637ff2a3-981c-497b-96b7-d72fa70dfd2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd15ad002cb914f14866e7608b67d305b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:21.510655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cd189850-4e4d-4af7-afd6-c31455f15d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:21.526856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd189850-4e4d-4af7-afd6-c31455f15d35 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:21.531054 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-637ff2a3-981c-497b-96b7-d72fa70dfd2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd15ad002cb914f14866e7608b67d305b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:21.534958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f18f47-fe0e-4307-ae1a-5d2baf3b6dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:21.535650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f18f47-fe0e-4307-ae1a-5d2baf3b6dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:21.536260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f18f47-fe0e-4307-ae1a-5d2baf3b6dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:21.536741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f18f47-fe0e-4307-ae1a-5d2baf3b6dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:21.538852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f18f47-fe0e-4307-ae1a-5d2baf3b6dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:21.540757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f18f47-fe0e-4307-ae1a-5d2baf3b6dd8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:21.544593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:21.551319 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-637ff2a3-981c-497b-96b7-d72fa70dfd2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd15ad002cb914f14866e7608b67d305b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:21.578134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 950/1838] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:21 2022] POST /identity/v3/projects => generated 401 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:21.582132 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f18f47-fe0e-4307-ae1a-5d2baf3b6dd8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:21.585086 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-28f18f47-fe0e-4307-ae1a-5d2baf3b6dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:21.604282 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 889/1839] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:21 2022] DELETE /identity/v3/users/d15ad002cb914f14866e7608b67d305b => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:21.613388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e9a98-97b6-4680-ad22-9cd6ec898efb None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:21.613388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e9a98-97b6-4680-ad22-9cd6ec898efb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:21.613668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e9a98-97b6-4680-ad22-9cd6ec898efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:21.655474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-797e9a98-97b6-4680-ad22-9cd6ec898efb None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:21.665084 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-797e9a98-97b6-4680-ad22-9cd6ec898efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:21.750570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f18f47-fe0e-4307-ae1a-5d2baf3b6dd8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:21.761431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc4225-1465-428a-a24c-a5482f87355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:21.762754 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc4225-1465-428a-a24c-a5482f87355f admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:21.763206 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc4225-1465-428a-a24c-a5482f87355f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:21.763569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc4225-1465-428a-a24c-a5482f87355f admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:21.764524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 890/1840] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:21.764633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc4225-1465-428a-a24c-a5482f87355f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:21.766773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc4225-1465-428a-a24c-a5482f87355f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:21.775190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e9a98-97b6-4680-ad22-9cd6ec898efb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:21.777999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-797e9a98-97b6-4680-ad22-9cd6ec898efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:21.780106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:21.801542 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 951/1841] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:21 2022] POST /identity/v3/users => generated 301 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:21.806106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc4225-1465-428a-a24c-a5482f87355f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:21.825203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc4225-1465-428a-a24c-a5482f87355f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:21.836839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa5e7d-524e-4786-9f70-fb1067b03fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:21.837535 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa5e7d-524e-4786-9f70-fb1067b03fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:21.837928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa5e7d-524e-4786-9f70-fb1067b03fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:21.838547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa5e7d-524e-4786-9f70-fb1067b03fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:21.839298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa5e7d-524e-4786-9f70-fb1067b03fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:21.841347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa5e7d-524e-4786-9f70-fb1067b03fd7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:21.860714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 952/1842] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:21.862807 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa5e7d-524e-4786-9f70-fb1067b03fd7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:21.879104 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e9a98-97b6-4680-ad22-9cd6ec898efb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:21.883894 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa5e7d-524e-4786-9f70-fb1067b03fd7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:21.886830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3e269-4e3f-486c-ba8d-dadbbf9e7be2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:21.887655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3e269-4e3f-486c-ba8d-dadbbf9e7be2 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:21.887951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3e269-4e3f-486c-ba8d-dadbbf9e7be2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:21.888238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be3e269-4e3f-486c-ba8d-dadbbf9e7be2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:21.890050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e8a5a-d93a-43b9-b777-46ecd5902924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:21.890861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e8a5a-d93a-43b9-b777-46ecd5902924 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:21.891250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e8a5a-d93a-43b9-b777-46ecd5902924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:21.891797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e8a5a-d93a-43b9-b777-46ecd5902924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:21.892693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e8a5a-d93a-43b9-b777-46ecd5902924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:21.894571 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e8a5a-d93a-43b9-b777-46ecd5902924 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:21.903090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3e269-4e3f-486c-ba8d-dadbbf9e7be2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:21.905276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be3e269-4e3f-486c-ba8d-dadbbf9e7be2 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:21.910808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 953/1843] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:21.910808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e8a5a-d93a-43b9-b777-46ecd5902924 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:21.913504 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0f0e8a5a-d93a-43b9-b777-46ecd5902924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:21.920913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 891/1844] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:21.925278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3e269-4e3f-486c-ba8d-dadbbf9e7be2 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:21.931250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e8a5a-d93a-43b9-b777-46ecd5902924 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:21.938285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b14f7-d561-4a79-bae1-edad8e201747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:21.940220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be3e269-4e3f-486c-ba8d-dadbbf9e7be2 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:21.940612 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b14f7-d561-4a79-bae1-edad8e201747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:21.941006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b14f7-d561-4a79-bae1-edad8e201747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:21.941384 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b14f7-d561-4a79-bae1-edad8e201747 admin admin] PATH_INFO: `/v3/projects/6ef267d9d488405caaa1cc8f08815f23` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:21.943551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b14f7-d561-4a79-bae1-edad8e201747 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ef267d9d488405caaa1cc8f08815f23)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:21.946185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842b14f7-d561-4a79-bae1-edad8e201747 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:21.951555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d7a6c-d41f-4d2d-8154-376b619e9c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:21.953605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d7a6c-d41f-4d2d-8154-376b619e9c7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:21.954550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d7a6c-d41f-4d2d-8154-376b619e9c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:21.959940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d7a6c-d41f-4d2d-8154-376b619e9c7f admin admin] PATH_INFO: `/v3/projects/68dbb92c4e934c2f93f23a11e6f75e24/users/f114b074822e40f2bdb0a28b6adfc3da/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:21.985322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d7a6c-d41f-4d2d-8154-376b619e9c7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=68dbb92c4e934c2f93f23a11e6f75e24, user_id=f114b074822e40f2bdb0a28b6adfc3da, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:21.990717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d7a6c-d41f-4d2d-8154-376b619e9c7f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:22.012968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 892/1845] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:21 2022] PUT /identity/v3/projects/68dbb92c4e934c2f93f23a11e6f75e24/users/f114b074822e40f2bdb0a28b6adfc3da/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:22.023637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3deb54-f674-4bc4-a396-959947aef39b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:22.023938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3deb54-f674-4bc4-a396-959947aef39b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:22.024219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3deb54-f674-4bc4-a396-959947aef39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:22.037404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-842b14f7-d561-4a79-bae1-edad8e201747 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ef267d9d488405caaa1cc8f08815f23', 'request_id': 'req-842b14f7-d561-4a79-bae1-edad8e201747'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:22.050164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-842b14f7-d561-4a79-bae1-edad8e201747 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ef267d9d488405caaa1cc8f08815f23', 'request_id': 'req-842b14f7-d561-4a79-bae1-edad8e201747'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:22.078528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-842b14f7-d561-4a79-bae1-edad8e201747 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ef267d9d488405caaa1cc8f08815f23', 'request_id': 'req-842b14f7-d561-4a79-bae1-edad8e201747'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:22.091027 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0b3deb54-f674-4bc4-a396-959947aef39b None None] MFA Rules not processed for user `f114b074822e40f2bdb0a28b6adfc3da`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:22.093645 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0b3deb54-f674-4bc4-a396-959947aef39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:22.108497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-842b14f7-d561-4a79-bae1-edad8e201747 admin admin] The token cache is being invalidate because project 6ef267d9d488405caaa1cc8f08815f23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:22.108833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-842b14f7-d561-4a79-bae1-edad8e201747 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:22.112501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-842b14f7-d561-4a79-bae1-edad8e201747 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:22.114773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 954/1846] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:21 2022] DELETE /identity/v3/projects/6ef267d9d488405caaa1cc8f08815f23 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:22.120586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931c1df-f956-42a8-98c5-17d45a986460 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:22.120912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931c1df-f956-42a8-98c5-17d45a986460 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:22.121252 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931c1df-f956-42a8-98c5-17d45a986460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:22.142568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f931c1df-f956-42a8-98c5-17d45a986460 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:22.144291 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f931c1df-f956-42a8-98c5-17d45a986460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:22.243347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 955/1847] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:22.249669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce353b-d664-4a2f-8567-4b1ccb89a3fc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:22.249961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce353b-d664-4a2f-8567-4b1ccb89a3fc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:22.250241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce353b-d664-4a2f-8567-4b1ccb89a3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:22.261472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:22.279715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 893/1848] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:22.280733 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3deb54-f674-4bc4-a396-959947aef39b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:22.284127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0b3deb54-f674-4bc4-a396-959947aef39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:22.293539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-dcce353b-d664-4a2f-8567-4b1ccb89a3fc None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:22.295056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-dcce353b-d664-4a2f-8567-4b1ccb89a3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:22.313892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3deb54-f674-4bc4-a396-959947aef39b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:22.316855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 956/1849] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:22.320379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce353b-d664-4a2f-8567-4b1ccb89a3fc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:22.329294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554b1ff-a9ff-42b1-8907-c74333f05a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:22.329906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554b1ff-a9ff-42b1-8907-c74333f05a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:22.330471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554b1ff-a9ff-42b1-8907-c74333f05a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:22.330759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554b1ff-a9ff-42b1-8907-c74333f05a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:22.332934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554b1ff-a9ff-42b1-8907-c74333f05a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:22.337300 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554b1ff-a9ff-42b1-8907-c74333f05a8c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:22.342721 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce353b-d664-4a2f-8567-4b1ccb89a3fc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:22.356385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d2fbb-d38d-4ff9-893a-7f3d18346afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:22.357499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d2fbb-d38d-4ff9-893a-7f3d18346afe admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:22.357795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d2fbb-d38d-4ff9-893a-7f3d18346afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:22.358081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d2fbb-d38d-4ff9-893a-7f3d18346afe admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:22.359117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d2fbb-d38d-4ff9-893a-7f3d18346afe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:22.359354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d2fbb-d38d-4ff9-893a-7f3d18346afe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:22.364193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d2fbb-d38d-4ff9-893a-7f3d18346afe admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:22.398014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 957/1850] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:22.399274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 894/1851] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:22 2022] POST /identity/v3/projects => generated 399 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:22.402164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d2fbb-d38d-4ff9-893a-7f3d18346afe admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:22.403224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554b1ff-a9ff-42b1-8907-c74333f05a8c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:22.432103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d2fbb-d38d-4ff9-893a-7f3d18346afe admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:22.492731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:22.552282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c40abe-16d1-4a6a-9fb7-2f75aee232d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:22.553150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c40abe-16d1-4a6a-9fb7-2f75aee232d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:22.553450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c40abe-16d1-4a6a-9fb7-2f75aee232d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:22.553766 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c40abe-16d1-4a6a-9fb7-2f75aee232d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:22.557409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c40abe-16d1-4a6a-9fb7-2f75aee232d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:22.561068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c40abe-16d1-4a6a-9fb7-2f75aee232d3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:22.592878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554b1ff-a9ff-42b1-8907-c74333f05a8c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:22.615467 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513aa04-d339-45c2-86fe-5b49059df05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:22.616552 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513aa04-d339-45c2-86fe-5b49059df05f admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:22.616940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513aa04-d339-45c2-86fe-5b49059df05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:22.617289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f513aa04-d339-45c2-86fe-5b49059df05f admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:22.618073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513aa04-d339-45c2-86fe-5b49059df05f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:22.620159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f513aa04-d339-45c2-86fe-5b49059df05f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:22.623449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 958/1852] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:22 2022] POST /identity/v3/projects => generated 369 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:22.624357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c40abe-16d1-4a6a-9fb7-2f75aee232d3 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:22.633742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e2c40abe-16d1-4a6a-9fb7-2f75aee232d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:22.648765 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 895/1853] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:22 2022] POST /identity/v3/users => generated 306 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:22.650065 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513aa04-d339-45c2-86fe-5b49059df05f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:22.676108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f513aa04-d339-45c2-86fe-5b49059df05f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:22.755532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d77f7-7f0c-4812-bd53-309e407806f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:22.756417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d77f7-7f0c-4812-bd53-309e407806f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:22.757416 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d77f7-7f0c-4812-bd53-309e407806f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:22.757823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d77f7-7f0c-4812-bd53-309e407806f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:22.758678 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d77f7-7f0c-4812-bd53-309e407806f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:22.761493 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d77f7-7f0c-4812-bd53-309e407806f2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:22.792971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 896/1854] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:22 2022] POST /identity/v3/users => generated 285 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:22.794058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d77f7-7f0c-4812-bd53-309e407806f2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:22.816080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d77f7-7f0c-4812-bd53-309e407806f2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:22.825280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c40abe-16d1-4a6a-9fb7-2f75aee232d3 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:22.831817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:22.833059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:22.833665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:22.834177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9014c2e-6331-404f-8670-c3413c9fcd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:22.834301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] PATH_INFO: `/v3/projects/ca4bf7461ca846e4871dc1a793dd0dc8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:22.835422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9014c2e-6331-404f-8670-c3413c9fcd44 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:22.835727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9014c2e-6331-404f-8670-c3413c9fcd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:22.836010 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9014c2e-6331-404f-8670-c3413c9fcd44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:22.836789 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9014c2e-6331-404f-8670-c3413c9fcd44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:22.837390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca4bf7461ca846e4871dc1a793dd0dc8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:22.839554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:22.840827 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9014c2e-6331-404f-8670-c3413c9fcd44 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:22.867218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 897/1855] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:22.867807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9014c2e-6331-404f-8670-c3413c9fcd44 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:22.887388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9014c2e-6331-404f-8670-c3413c9fcd44 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:22.908929 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca4bf7461ca846e4871dc1a793dd0dc8', 'request_id': 'req-cc7a70d1-8847-4770-932b-a77389d17598'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:22.924365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca4bf7461ca846e4871dc1a793dd0dc8', 'request_id': 'req-cc7a70d1-8847-4770-932b-a77389d17598'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:22.940956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca4bf7461ca846e4871dc1a793dd0dc8', 'request_id': 'req-cc7a70d1-8847-4770-932b-a77389d17598'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:22.977728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] The token cache is being invalidate because project ca4bf7461ca846e4871dc1a793dd0dc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:22.979905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:22.980702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:22.982266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:22.983565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 959/1856] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:22 2022] DELETE /identity/v3/projects/ca4bf7461ca846e4871dc1a793dd0dc8 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:22.984552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:22.986726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:23.040825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdc141-b9c3-4b91-bc52-c7404f6ad5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:23.041537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdc141-b9c3-4b91-bc52-c7404f6ad5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:23.041828 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdc141-b9c3-4b91-bc52-c7404f6ad5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:23.042112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbdc141-b9c3-4b91-bc52-c7404f6ad5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:23.042851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdc141-b9c3-4b91-bc52-c7404f6ad5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:23.044812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbdc141-b9c3-4b91-bc52-c7404f6ad5d1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:23.072566 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 898/1857] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:23.073214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdc141-b9c3-4b91-bc52-c7404f6ad5d1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:23.079273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8cbdc141-b9c3-4b91-bc52-c7404f6ad5d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:23.090973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:23.104557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbdc141-b9c3-4b91-bc52-c7404f6ad5d1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:23.119404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:23.120310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:23.120608 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:23.120928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] PATH_INFO: `/v3/users/68204bed43344af9ba7a83c6786589b8` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:23.123924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] RBAC: Authorizing `identity:delete_user(user_id=68204bed43344af9ba7a83c6786589b8)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:23.125399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:23.130893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a70d1-8847-4770-932b-a77389d17598 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:23.147424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a31b57-7267-416d-896f-c568ecb2f39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:23.148654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a31b57-7267-416d-896f-c568ecb2f39c admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:23.148975 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a31b57-7267-416d-896f-c568ecb2f39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:23.154292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a31b57-7267-416d-896f-c568ecb2f39c admin admin] PATH_INFO: `/v3/projects/a9a40e2e991645589ba74cf66112628a/users/63aaebcb56cf48dc8d50d6a0c16fb1eb/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:23.197413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a31b57-7267-416d-896f-c568ecb2f39c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a40e2e991645589ba74cf66112628a, user_id=63aaebcb56cf48dc8d50d6a0c16fb1eb, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:23.207760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a31b57-7267-416d-896f-c568ecb2f39c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:23.223292 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68204bed43344af9ba7a83c6786589b8'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:23.224081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68204bed43344af9ba7a83c6786589b8'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:23.231417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68204bed43344af9ba7a83c6786589b8'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:23.238418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 960/1858] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:22 2022] PUT /identity/v3/projects/a9a40e2e991645589ba74cf66112628a/users/63aaebcb56cf48dc8d50d6a0c16fb1eb/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:23.238418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816216b5-b83c-4291-8091-f40b7b28c3fc None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:23.238418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816216b5-b83c-4291-8091-f40b7b28c3fc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:23.238738 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816216b5-b83c-4291-8091-f40b7b28c3fc None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:23.245187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 961/1859] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 11:55:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:55:23.247173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816216b5-b83c-4291-8091-f40b7b28c3fc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:23.251563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-816216b5-b83c-4291-8091-f40b7b28c3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:23.258686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68204bed43344af9ba7a83c6786589b8'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:23.269871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68204bed43344af9ba7a83c6786589b8'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:23.298874 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 899/1860] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:23 2022] DELETE /identity/v3/users/68204bed43344af9ba7a83c6786589b8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:23.299911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:23.301598 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:23.382715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9c9d0-3fbe-4128-875b-7b685f8ba30c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:23.389239 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5866ab48-2f42-47a9-8514-b5a7e04e762d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:23.389929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866ab48-2f42-47a9-8514-b5a7e04e762d admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:23.390220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866ab48-2f42-47a9-8514-b5a7e04e762d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:23.390502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5866ab48-2f42-47a9-8514-b5a7e04e762d admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:23.391244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866ab48-2f42-47a9-8514-b5a7e04e762d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:23.392899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5866ab48-2f42-47a9-8514-b5a7e04e762d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:23.394044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816216b5-b83c-4291-8091-f40b7b28c3fc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:23.401218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139874b4-01d1-450b-b491-b91e05353da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:23.402310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139874b4-01d1-450b-b491-b91e05353da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:23.403025 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139874b4-01d1-450b-b491-b91e05353da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:23.403569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139874b4-01d1-450b-b491-b91e05353da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:23.404702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139874b4-01d1-450b-b491-b91e05353da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:23.406472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 900/1861] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:23.407266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139874b4-01d1-450b-b491-b91e05353da5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:23.407788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eef314-3b2a-456e-9ed4-2788e2e9cb0b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:23.408084 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eef314-3b2a-456e-9ed4-2788e2e9cb0b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:23.408367 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eef314-3b2a-456e-9ed4-2788e2e9cb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:23.431839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 962/1862] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:23 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:23.442068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-53eef314-3b2a-456e-9ed4-2788e2e9cb0b None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:23.445456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-53eef314-3b2a-456e-9ed4-2788e2e9cb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:23.455873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139874b4-01d1-450b-b491-b91e05353da5 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:23.476216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139874b4-01d1-450b-b491-b91e05353da5 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:23.481028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5661019-0da6-40cf-91e1-709ea562e53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:23.495767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5661019-0da6-40cf-91e1-709ea562e53c admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:23.496425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5661019-0da6-40cf-91e1-709ea562e53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:23.496630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5661019-0da6-40cf-91e1-709ea562e53c admin admin] PATH_INFO: `/v3/projects/a9a40e2e991645589ba74cf66112628a/users/63aaebcb56cf48dc8d50d6a0c16fb1eb/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:23.502608 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5661019-0da6-40cf-91e1-709ea562e53c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a40e2e991645589ba74cf66112628a, user_id=63aaebcb56cf48dc8d50d6a0c16fb1eb, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:23.508810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5661019-0da6-40cf-91e1-709ea562e53c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:23.525686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:23.551134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 963/1863] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:23 2022] PUT /identity/v3/projects/a9a40e2e991645589ba74cf66112628a/users/63aaebcb56cf48dc8d50d6a0c16fb1eb/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:23.552172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5661019-0da6-40cf-91e1-709ea562e53c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:23.575087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 901/1864] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 11:55:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:23.576985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f380d-3832-4c10-aa5b-ca3839d9d449 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:23.577471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f380d-3832-4c10-aa5b-ca3839d9d449 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:23.577940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f380d-3832-4c10-aa5b-ca3839d9d449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:23.626771 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5661019-0da6-40cf-91e1-709ea562e53c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:23.644811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bb431-6f09-401d-b73f-81e6b923861d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:23.647386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb431-6f09-401d-b73f-81e6b923861d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:23.647905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb431-6f09-401d-b73f-81e6b923861d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:23.648057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb431-6f09-401d-b73f-81e6b923861d admin admin] PATH_INFO: `/v3/projects/3b88aebfb7d34bc8b30f40080dc23d32/users/2c2373065039411ea68793775c729d0c/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:23.663501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-5e1f380d-3832-4c10-aa5b-ca3839d9d449 None None] MFA Rules not processed for user `63aaebcb56cf48dc8d50d6a0c16fb1eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:23.665183 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5e1f380d-3832-4c10-aa5b-ca3839d9d449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:23.687147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bb431-6f09-401d-b73f-81e6b923861d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b88aebfb7d34bc8b30f40080dc23d32, user_id=2c2373065039411ea68793775c729d0c, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:23.693172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bb431-6f09-401d-b73f-81e6b923861d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:23.752224 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 964/1865] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:23 2022] PUT /identity/v3/projects/3b88aebfb7d34bc8b30f40080dc23d32/users/2c2373065039411ea68793775c729d0c/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:23.765171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53604025-c49f-40b3-b7df-dcfa15b5ed2c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:23.766159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53604025-c49f-40b3-b7df-dcfa15b5ed2c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:23.766159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53604025-c49f-40b3-b7df-dcfa15b5ed2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:23.872636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-53604025-c49f-40b3-b7df-dcfa15b5ed2c None None] MFA Rules not processed for user `2c2373065039411ea68793775c729d0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:23.875791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-53604025-c49f-40b3-b7df-dcfa15b5ed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:23.993709 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:23.999998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 902/1866] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:23 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:24.002627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f380d-3832-4c10-aa5b-ca3839d9d449 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:24.008893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5e1f380d-3832-4c10-aa5b-ca3839d9d449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:24.072970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 965/1867] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:24.074110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53604025-c49f-40b3-b7df-dcfa15b5ed2c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:24.077179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-53604025-c49f-40b3-b7df-dcfa15b5ed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:24.121735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f380d-3832-4c10-aa5b-ca3839d9d449 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:24.163574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53604025-c49f-40b3-b7df-dcfa15b5ed2c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:24.168205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab3615-3fed-4f46-9766-cb0f9ad8a0b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:24.169631 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab3615-3fed-4f46-9766-cb0f9ad8a0b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:24.170256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab3615-3fed-4f46-9766-cb0f9ad8a0b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:24.170768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab3615-3fed-4f46-9766-cb0f9ad8a0b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:24.211922 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:24.223360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1675b-e089-45dc-9821-718aace0bfcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:24.224036 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1675b-e089-45dc-9821-718aace0bfcf service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:24.224979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1675b-e089-45dc-9821-718aace0bfcf service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:24.225203 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1675b-e089-45dc-9821-718aace0bfcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:24.247192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1675b-e089-45dc-9821-718aace0bfcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:24.249199 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1675b-e089-45dc-9821-718aace0bfcf service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:24.262904 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 903/1868] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:24.263927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1675b-e089-45dc-9821-718aace0bfcf service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:24.269899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab3615-3fed-4f46-9766-cb0f9ad8a0b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:24.271506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab3615-3fed-4f46-9766-cb0f9ad8a0b8 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:24.285382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 966/1869] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:24 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:24.404991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1675b-e089-45dc-9821-718aace0bfcf service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:24.412802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab3615-3fed-4f46-9766-cb0f9ad8a0b8 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:24.417653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa5d9c-aec4-4c9d-9171-a0b63e9e695b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:24.419344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa5d9c-aec4-4c9d-9171-a0b63e9e695b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:24.419642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa5d9c-aec4-4c9d-9171-a0b63e9e695b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:24.419990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa5d9c-aec4-4c9d-9171-a0b63e9e695b admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:24.420798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa5d9c-aec4-4c9d-9171-a0b63e9e695b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:24.425413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa5d9c-aec4-4c9d-9171-a0b63e9e695b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:24.431541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab3615-3fed-4f46-9766-cb0f9ad8a0b8 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:24.436983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad75b2-35a8-4382-9b0e-47f24134c914 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:24.438730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad75b2-35a8-4382-9b0e-47f24134c914 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:24.440218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad75b2-35a8-4382-9b0e-47f24134c914 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:24.440756 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad75b2-35a8-4382-9b0e-47f24134c914 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:24.443967 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f6ad75b2-35a8-4382-9b0e-47f24134c914 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:24.504443 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 904/1870] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:24 2022] POST /identity/v3/projects => generated 369 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:24.505684 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa5d9c-aec4-4c9d-9171-a0b63e9e695b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:24.541562 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad75b2-35a8-4382-9b0e-47f24134c914 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:24.544262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad75b2-35a8-4382-9b0e-47f24134c914 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:24.558365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 967/1871] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:24.559519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad75b2-35a8-4382-9b0e-47f24134c914 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:24.561676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f6ad75b2-35a8-4382-9b0e-47f24134c914 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:24.582351 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad75b2-35a8-4382-9b0e-47f24134c914 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:24.598280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:24.615501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa5d9c-aec4-4c9d-9171-a0b63e9e695b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:24.624222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07e5db-a6a2-4be2-8543-65c10ad791cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:24.625147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07e5db-a6a2-4be2-8543-65c10ad791cc admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:24.626556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07e5db-a6a2-4be2-8543-65c10ad791cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:24.627083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af07e5db-a6a2-4be2-8543-65c10ad791cc admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:24.628330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07e5db-a6a2-4be2-8543-65c10ad791cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:24.649683 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af07e5db-a6a2-4be2-8543-65c10ad791cc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:24.672001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3677e92c-221b-4a08-b1f3-8e04d9a2c196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:24.673112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677e92c-221b-4a08-b1f3-8e04d9a2c196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:24.673405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677e92c-221b-4a08-b1f3-8e04d9a2c196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:24.673686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3677e92c-221b-4a08-b1f3-8e04d9a2c196 admin admin] PATH_INFO: `/v3/projects/418bba389a8249f8ad93cf79d589d968` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:24.679170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677e92c-221b-4a08-b1f3-8e04d9a2c196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=418bba389a8249f8ad93cf79d589d968)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:24.682215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3677e92c-221b-4a08-b1f3-8e04d9a2c196 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:24.706086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 905/1872] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:24 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:24.710747 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07e5db-a6a2-4be2-8543-65c10ad791cc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:24.731321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af07e5db-a6a2-4be2-8543-65c10ad791cc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:24.744699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b852a-c5d0-4a51-8785-43f9c9ce6c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:24.745922 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b852a-c5d0-4a51-8785-43f9c9ce6c8d admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:24.746583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b852a-c5d0-4a51-8785-43f9c9ce6c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:24.747029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b852a-c5d0-4a51-8785-43f9c9ce6c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:24.747945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b852a-c5d0-4a51-8785-43f9c9ce6c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:24.753638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425b852a-c5d0-4a51-8785-43f9c9ce6c8d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:24.755118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3677e92c-221b-4a08-b1f3-8e04d9a2c196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '418bba389a8249f8ad93cf79d589d968', 'request_id': 'req-3677e92c-221b-4a08-b1f3-8e04d9a2c196'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:24.769194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3677e92c-221b-4a08-b1f3-8e04d9a2c196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '418bba389a8249f8ad93cf79d589d968', 'request_id': 'req-3677e92c-221b-4a08-b1f3-8e04d9a2c196'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:24.782641 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 906/1873] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:24.788338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b852a-c5d0-4a51-8785-43f9c9ce6c8d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:24.791984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3677e92c-221b-4a08-b1f3-8e04d9a2c196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '418bba389a8249f8ad93cf79d589d968', 'request_id': 'req-3677e92c-221b-4a08-b1f3-8e04d9a2c196'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:24.810642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b852a-c5d0-4a51-8785-43f9c9ce6c8d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:24.817811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3677e92c-221b-4a08-b1f3-8e04d9a2c196 admin admin] The token cache is being invalidate because project 418bba389a8249f8ad93cf79d589d968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:24.818438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3677e92c-221b-4a08-b1f3-8e04d9a2c196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:24.824374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3677e92c-221b-4a08-b1f3-8e04d9a2c196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:24.828378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 968/1874] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:24 2022] DELETE /identity/v3/projects/418bba389a8249f8ad93cf79d589d968 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:24.835505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e7d79-cd06-4026-9f03-162fa8302f99 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:24.836998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e7d79-cd06-4026-9f03-162fa8302f99 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:24.837671 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e7d79-cd06-4026-9f03-162fa8302f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:24.878045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ce7e7d79-cd06-4026-9f03-162fa8302f99 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:24.880084 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ce7e7d79-cd06-4026-9f03-162fa8302f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:24.950909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:24.969377 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd123d-5149-4cb3-ac5c-9528907ae49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:24.971320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd123d-5149-4cb3-ac5c-9528907ae49c admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:24.972103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd123d-5149-4cb3-ac5c-9528907ae49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:24.973278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd123d-5149-4cb3-ac5c-9528907ae49c admin admin] PATH_INFO: `/v3/projects/5e0a9ccdd1f34556a78af4735e9f4813/users/ec36ae80eb9a4ea9ab6866e7bb55510f/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:24.992606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 969/1875] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:25.005119 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c8505-9749-440e-86b8-34b10009c00f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:25.005299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c8505-9749-440e-86b8-34b10009c00f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:25.005577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c8505-9749-440e-86b8-34b10009c00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:25.048473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd123d-5149-4cb3-ac5c-9528907ae49c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e0a9ccdd1f34556a78af4735e9f4813, user_id=ec36ae80eb9a4ea9ab6866e7bb55510f, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:25.052349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd123d-5149-4cb3-ac5c-9528907ae49c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:25.060854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-be4c8505-9749-440e-86b8-34b10009c00f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:25.063287 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-be4c8505-9749-440e-86b8-34b10009c00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:25.094336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 970/1876] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:25.097854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c8505-9749-440e-86b8-34b10009c00f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:25.103246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 907/1877] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:24 2022] PUT /identity/v3/projects/5e0a9ccdd1f34556a78af4735e9f4813/users/ec36ae80eb9a4ea9ab6866e7bb55510f/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:25.105002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd123d-5149-4cb3-ac5c-9528907ae49c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:25.107242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1bcd123d-5149-4cb3-ac5c-9528907ae49c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:25.120837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c8505-9749-440e-86b8-34b10009c00f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:25.232653 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:25.285333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a059427-92fd-4366-8f58-5e01da4473e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:25.287277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a059427-92fd-4366-8f58-5e01da4473e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:25.287579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a059427-92fd-4366-8f58-5e01da4473e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:25.288016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a059427-92fd-4366-8f58-5e01da4473e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:25.290351 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a059427-92fd-4366-8f58-5e01da4473e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:25.291685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd123d-5149-4cb3-ac5c-9528907ae49c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:25.294644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a059427-92fd-4366-8f58-5e01da4473e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:25.300229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a059427-92fd-4366-8f58-5e01da4473e0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:25.308082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbf976-810c-4e3c-9acb-db670d544f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:25.326442 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbf976-810c-4e3c-9acb-db670d544f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:25.326775 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbf976-810c-4e3c-9acb-db670d544f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:25.327060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbf976-810c-4e3c-9acb-db670d544f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:25.327843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbf976-810c-4e3c-9acb-db670d544f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:25.340606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbf976-810c-4e3c-9acb-db670d544f7f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:25.370252 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 971/1878] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:25.372393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a059427-92fd-4366-8f58-5e01da4473e0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:25.380622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 908/1879] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:25.384927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbf976-810c-4e3c-9acb-db670d544f7f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:25.398118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a059427-92fd-4366-8f58-5e01da4473e0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:25.409786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbf976-810c-4e3c-9acb-db670d544f7f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:25.414210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103043d-ca82-4efe-a659-7f64fa33228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:25.415518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103043d-ca82-4efe-a659-7f64fa33228d admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:25.415518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103043d-ca82-4efe-a659-7f64fa33228d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:25.415518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103043d-ca82-4efe-a659-7f64fa33228d admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:25.416126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103043d-ca82-4efe-a659-7f64fa33228d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:25.419778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103043d-ca82-4efe-a659-7f64fa33228d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:25.425980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec60103-472a-4ee4-b5ce-919433325d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:25.427036 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec60103-472a-4ee4-b5ce-919433325d6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:25.427574 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec60103-472a-4ee4-b5ce-919433325d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:25.427866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec60103-472a-4ee4-b5ce-919433325d6a admin admin] PATH_INFO: `/v3/projects/5e0a9ccdd1f34556a78af4735e9f4813/users/ec36ae80eb9a4ea9ab6866e7bb55510f/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:25.434400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec60103-472a-4ee4-b5ce-919433325d6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e0a9ccdd1f34556a78af4735e9f4813, user_id=ec36ae80eb9a4ea9ab6866e7bb55510f, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:25.442411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec60103-472a-4ee4-b5ce-919433325d6a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:25.467269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 909/1880] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:25 2022] PUT /identity/v3/projects/5e0a9ccdd1f34556a78af4735e9f4813/users/ec36ae80eb9a4ea9ab6866e7bb55510f/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:25.488236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8fcab-8f98-41da-9ee5-54de581c6e9e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:25.488832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8fcab-8f98-41da-9ee5-54de581c6e9e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:25.490145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8fcab-8f98-41da-9ee5-54de581c6e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:25.502594 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 972/1881] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:25 2022] POST /identity/v3/projects => generated 359 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:25.507043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103043d-ca82-4efe-a659-7f64fa33228d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:25.575993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ebc8fcab-8f98-41da-9ee5-54de581c6e9e None None] MFA Rules not processed for user `ec36ae80eb9a4ea9ab6866e7bb55510f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:25.579002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ebc8fcab-8f98-41da-9ee5-54de581c6e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:25.696396 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103043d-ca82-4efe-a659-7f64fa33228d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:25.715371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb809a3d-2aaa-41dd-b935-8d267eaaf398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:25.716417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb809a3d-2aaa-41dd-b935-8d267eaaf398 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:25.717533 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb809a3d-2aaa-41dd-b935-8d267eaaf398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:25.717885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb809a3d-2aaa-41dd-b935-8d267eaaf398 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:25.719277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb809a3d-2aaa-41dd-b935-8d267eaaf398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:25.725549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb809a3d-2aaa-41dd-b935-8d267eaaf398 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:25.775898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 973/1882] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:25 2022] POST /identity/v3/users => generated 280 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:25.777410 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb809a3d-2aaa-41dd-b935-8d267eaaf398 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:25.793219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb809a3d-2aaa-41dd-b935-8d267eaaf398 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:25.802167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15778c-105f-49ba-b48f-adfae1f91ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:25.802922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15778c-105f-49ba-b48f-adfae1f91ecf admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:25.804151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15778c-105f-49ba-b48f-adfae1f91ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:25.804576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d15778c-105f-49ba-b48f-adfae1f91ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:25.805770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15778c-105f-49ba-b48f-adfae1f91ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:25.807331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 910/1883] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:25 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:25.807832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d15778c-105f-49ba-b48f-adfae1f91ecf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:25.820846 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8fcab-8f98-41da-9ee5-54de581c6e9e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:25.822593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ebc8fcab-8f98-41da-9ee5-54de581c6e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:25.826344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 974/1884] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:25.829175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15778c-105f-49ba-b48f-adfae1f91ecf admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:25.848717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d15778c-105f-49ba-b48f-adfae1f91ecf admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:25.860488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f6ea2-91c9-4588-9063-b7a7166d17e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:25.861387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f6ea2-91c9-4588-9063-b7a7166d17e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:25.861502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f6ea2-91c9-4588-9063-b7a7166d17e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:25.861795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f6ea2-91c9-4588-9063-b7a7166d17e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:25.862505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f6ea2-91c9-4588-9063-b7a7166d17e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:25.867646 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f6ea2-91c9-4588-9063-b7a7166d17e3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:25.887462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 975/1885] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:25.891672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f6ea2-91c9-4588-9063-b7a7166d17e3 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:25.905275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f6ea2-91c9-4588-9063-b7a7166d17e3 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:25.912516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47b8f9-da01-4d14-9cfb-1fb3516cb062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:25.914105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47b8f9-da01-4d14-9cfb-1fb3516cb062 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:25.914617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47b8f9-da01-4d14-9cfb-1fb3516cb062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:25.915100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47b8f9-da01-4d14-9cfb-1fb3516cb062 admin admin] PATH_INFO: `/v3/projects/169b01318f174ad8a607b8454c79ff02/users/0db2327b9c8b42dbaff11cbf181635b5/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:25.939146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47b8f9-da01-4d14-9cfb-1fb3516cb062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=169b01318f174ad8a607b8454c79ff02, user_id=0db2327b9c8b42dbaff11cbf181635b5, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:25.943547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47b8f9-da01-4d14-9cfb-1fb3516cb062 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:25.982391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:25.993369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 976/1886] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:25 2022] PUT /identity/v3/projects/169b01318f174ad8a607b8454c79ff02/users/0db2327b9c8b42dbaff11cbf181635b5/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:25.999942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f971fa-4980-437e-9d86-67ad8ede805c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:26.000247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f971fa-4980-437e-9d86-67ad8ede805c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:26.001553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f971fa-4980-437e-9d86-67ad8ede805c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:26.016818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8fcab-8f98-41da-9ee5-54de581c6e9e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:26.065413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-90f971fa-4980-437e-9d86-67ad8ede805c None None] MFA Rules not processed for user `0db2327b9c8b42dbaff11cbf181635b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:26.070938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-90f971fa-4980-437e-9d86-67ad8ede805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:26.079692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd0949-94ba-4941-b01d-6a065cee81df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:26.080395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd0949-94ba-4941-b01d-6a065cee81df service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:26.080878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd0949-94ba-4941-b01d-6a065cee81df service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:26.081213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd0949-94ba-4941-b01d-6a065cee81df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:26.175197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd0949-94ba-4941-b01d-6a065cee81df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:26.177333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd0949-94ba-4941-b01d-6a065cee81df service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:26.183286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:26.195275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 911/1887] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:25 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:26.255741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 977/1888] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:25 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:26.397732 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd0949-94ba-4941-b01d-6a065cee81df service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:26.399234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0bfd0949-94ba-4941-b01d-6a065cee81df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:26.423291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd0949-94ba-4941-b01d-6a065cee81df service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:26.437968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69576465-5a07-49cc-9774-c633d61cb83b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:26.438635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69576465-5a07-49cc-9774-c633d61cb83b service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:26.438927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69576465-5a07-49cc-9774-c633d61cb83b service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:26.439205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69576465-5a07-49cc-9774-c633d61cb83b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:26.477594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69576465-5a07-49cc-9774-c633d61cb83b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:26.479066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69576465-5a07-49cc-9774-c633d61cb83b service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:26.519789 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 912/1889] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:26 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:26.546572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f971fa-4980-437e-9d86-67ad8ede805c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:26.748948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f971fa-4980-437e-9d86-67ad8ede805c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:26.760634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:26.761561 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:26.763556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:26.763894 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] PATH_INFO: `/v3/users/0db2327b9c8b42dbaff11cbf181635b5` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:26.768147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0db2327b9c8b42dbaff11cbf181635b5)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:26.773095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:26.923667 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0db2327b9c8b42dbaff11cbf181635b5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:26.924283 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0db2327b9c8b42dbaff11cbf181635b5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:26.924973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0db2327b9c8b42dbaff11cbf181635b5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:26.950990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0db2327b9c8b42dbaff11cbf181635b5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:26.963404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0db2327b9c8b42dbaff11cbf181635b5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:26.994668 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 978/1890] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:26 2022] DELETE /identity/v3/users/0db2327b9c8b42dbaff11cbf181635b5 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:26.997328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba085b-2e76-4835-87b2-e545ba6a98b8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:26.997653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba085b-2e76-4835-87b2-e545ba6a98b8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:26.997948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba085b-2e76-4835-87b2-e545ba6a98b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:27.023619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-baba085b-2e76-4835-87b2-e545ba6a98b8 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:27.025761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-baba085b-2e76-4835-87b2-e545ba6a98b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:27.162482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 913/1891] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:27.211509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:27.215897 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:27.361627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281dc2-e3f7-4da4-9674-698f0e6d325a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:27.370040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452df2af-afe0-46de-9a57-cedf5a4b727e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:27.371796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452df2af-afe0-46de-9a57-cedf5a4b727e service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:27.372363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452df2af-afe0-46de-9a57-cedf5a4b727e service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:27.372656 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452df2af-afe0-46de-9a57-cedf5a4b727e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:27.395449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452df2af-afe0-46de-9a57-cedf5a4b727e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:27.400009 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452df2af-afe0-46de-9a57-cedf5a4b727e service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:27.423286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 979/1892] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:28.758159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba085b-2e76-4835-87b2-e545ba6a98b8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:28.759859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-baba085b-2e76-4835-87b2-e545ba6a98b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:28.788037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba085b-2e76-4835-87b2-e545ba6a98b8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:28.796975 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:28.797860 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:28.798145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:28.798496 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] PATH_INFO: `/v3/projects/169b01318f174ad8a607b8454c79ff02` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:28.800192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] RBAC: Authorizing `identity:delete_project(project_id=169b01318f174ad8a607b8454c79ff02)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:28.803399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:28.874326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:28.899486 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '169b01318f174ad8a607b8454c79ff02', 'request_id': 'req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:28.935167 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '169b01318f174ad8a607b8454c79ff02', 'request_id': 'req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:28.979101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '169b01318f174ad8a607b8454c79ff02', 'request_id': 'req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:28.999862 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] The token cache is being invalidate because project 169b01318f174ad8a607b8454c79ff02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:29.000539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:29.003077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:29.005367 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 914/1893] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:28 2022] DELETE /identity/v3/projects/169b01318f174ad8a607b8454c79ff02 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:29.011208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4637d98-9523-4c3e-aa53-a5b0bb1e1ec3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.011759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4637d98-9523-4c3e-aa53-a5b0bb1e1ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.011759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4637d98-9523-4c3e-aa53-a5b0bb1e1ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.030832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:29.033744 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:29.047424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b4637d98-9523-4c3e-aa53-a5b0bb1e1ec3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:29.048658 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b4637d98-9523-4c3e-aa53-a5b0bb1e1ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:29.062339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:29.171070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 980/1894] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:29.180309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4d6cf-9cdd-40ce-aace-7afd425970fa None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.181467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4d6cf-9cdd-40ce-aace-7afd425970fa None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.181961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab4d6cf-9cdd-40ce-aace-7afd425970fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.182431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4f05-e8e7-48f5-94a7-d8d4e42e5256 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:29.198130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8264f-370e-4ed9-afa6-2118587e4ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:29.199041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8264f-370e-4ed9-afa6-2118587e4ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.200501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8264f-370e-4ed9-afa6-2118587e4ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.201250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8264f-370e-4ed9-afa6-2118587e4ac2 admin admin] PATH_INFO: `/v3/users/2c2373065039411ea68793775c729d0c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.203706 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8264f-370e-4ed9-afa6-2118587e4ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c2373065039411ea68793775c729d0c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:29.205712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8264f-370e-4ed9-afa6-2118587e4ac2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:29.207956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-aab4d6cf-9cdd-40ce-aace-7afd425970fa None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:29.209512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-aab4d6cf-9cdd-40ce-aace-7afd425970fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:29.235573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 981/1895] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:29.242444 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4d6cf-9cdd-40ce-aace-7afd425970fa None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:29.254720 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab4d6cf-9cdd-40ce-aace-7afd425970fa None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:29.261946 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4473d-8a7d-4997-a50b-d13495d5538f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:29.263066 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4473d-8a7d-4997-a50b-d13495d5538f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.263838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4473d-8a7d-4997-a50b-d13495d5538f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.264327 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b4473d-8a7d-4997-a50b-d13495d5538f admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.266110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4473d-8a7d-4997-a50b-d13495d5538f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:29.266530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4473d-8a7d-4997-a50b-d13495d5538f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:29.268648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b4473d-8a7d-4997-a50b-d13495d5538f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:29.282682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 982/1896] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:55:29.289301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4473d-8a7d-4997-a50b-d13495d5538f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:29.315466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b4473d-8a7d-4997-a50b-d13495d5538f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:29.316708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-59b8264f-370e-4ed9-afa6-2118587e4ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c2373065039411ea68793775c729d0c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:29.317481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-59b8264f-370e-4ed9-afa6-2118587e4ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c2373065039411ea68793775c729d0c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:29.320057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-59b8264f-370e-4ed9-afa6-2118587e4ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c2373065039411ea68793775c729d0c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:29.325146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bb4bb-ffdc-46e1-8963-646ad87c32a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:29.326102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bb4bb-ffdc-46e1-8963-646ad87c32a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.326600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bb4bb-ffdc-46e1-8963-646ad87c32a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.327111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28bb4bb-ffdc-46e1-8963-646ad87c32a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.328015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bb4bb-ffdc-46e1-8963-646ad87c32a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:29.330367 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28bb4bb-ffdc-46e1-8963-646ad87c32a9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:29.341285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-59b8264f-370e-4ed9-afa6-2118587e4ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c2373065039411ea68793775c729d0c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:29.352115 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-59b8264f-370e-4ed9-afa6-2118587e4ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c2373065039411ea68793775c729d0c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:29.372061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 983/1897] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:29 2022] POST /identity/v3/projects => generated 379 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:29.372262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 915/1898] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:29 2022] DELETE /identity/v3/users/2c2373065039411ea68793775c729d0c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:29.372961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bb4bb-ffdc-46e1-8963-646ad87c32a9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:29.374369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da143ab1-8634-4ce0-807f-144491597f96 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.374669 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da143ab1-8634-4ce0-807f-144491597f96 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.374957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da143ab1-8634-4ce0-807f-144491597f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.375308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d28bb4bb-ffdc-46e1-8963-646ad87c32a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:29.393699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-da143ab1-8634-4ce0-807f-144491597f96 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:29.394947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-da143ab1-8634-4ce0-807f-144491597f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:29.438217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:29.494594 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28bb4bb-ffdc-46e1-8963-646ad87c32a9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:29.508205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 916/1899] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:29.525938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da143ab1-8634-4ce0-807f-144491597f96 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:29.529797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-da143ab1-8634-4ce0-807f-144491597f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:29.580247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324631dd-d910-42f1-9da9-78c260f67580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:29.582230 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324631dd-d910-42f1-9da9-78c260f67580 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.583052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324631dd-d910-42f1-9da9-78c260f67580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.584106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324631dd-d910-42f1-9da9-78c260f67580 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.591429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324631dd-d910-42f1-9da9-78c260f67580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:29.593547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324631dd-d910-42f1-9da9-78c260f67580 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:29.657972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 984/1900] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:29 2022] POST /identity/v3/users => generated 290 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:29.662129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324631dd-d910-42f1-9da9-78c260f67580 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:29.675263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:29.682246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324631dd-d910-42f1-9da9-78c260f67580 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:29.686228 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da143ab1-8634-4ce0-807f-144491597f96 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:29.690181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024d5a2-b62c-4511-a6cc-21730b616f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:29.690825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024d5a2-b62c-4511-a6cc-21730b616f21 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.691113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024d5a2-b62c-4511-a6cc-21730b616f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.691547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024d5a2-b62c-4511-a6cc-21730b616f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.692630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb51405-6262-496e-9cb8-99da2c74372b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:29.693775 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb51405-6262-496e-9cb8-99da2c74372b admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.694256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb51405-6262-496e-9cb8-99da2c74372b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.694679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb51405-6262-496e-9cb8-99da2c74372b admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.695560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb51405-6262-496e-9cb8-99da2c74372b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:29.700405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb51405-6262-496e-9cb8-99da2c74372b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:29.702228 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024d5a2-b62c-4511-a6cc-21730b616f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:29.703742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024d5a2-b62c-4511-a6cc-21730b616f21 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:29.718591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 985/1901] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:29.722878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb51405-6262-496e-9cb8-99da2c74372b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:29.737913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 917/1902] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:29.742820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb51405-6262-496e-9cb8-99da2c74372b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:29.744003 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024d5a2-b62c-4511-a6cc-21730b616f21 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:29.747388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8024d5a2-b62c-4511-a6cc-21730b616f21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:29.757112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd704378-6a0a-42ac-9228-73b3aa054d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:29.757948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd704378-6a0a-42ac-9228-73b3aa054d39 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.758233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd704378-6a0a-42ac-9228-73b3aa054d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.758516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd704378-6a0a-42ac-9228-73b3aa054d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.759768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd704378-6a0a-42ac-9228-73b3aa054d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:29.761468 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd704378-6a0a-42ac-9228-73b3aa054d39 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:29.781739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024d5a2-b62c-4511-a6cc-21730b616f21 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:29.791218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658e9d1-70d8-4ea0-9475-1563dff87e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:29.792396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658e9d1-70d8-4ea0-9475-1563dff87e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.792898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658e9d1-70d8-4ea0-9475-1563dff87e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.793433 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658e9d1-70d8-4ea0-9475-1563dff87e93 admin admin] PATH_INFO: `/v3/users/f114b074822e40f2bdb0a28b6adfc3da` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.794080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 986/1903] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:29 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:29.795782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658e9d1-70d8-4ea0-9475-1563dff87e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f114b074822e40f2bdb0a28b6adfc3da)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:29.798050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd704378-6a0a-42ac-9228-73b3aa054d39 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:29.799832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658e9d1-70d8-4ea0-9475-1563dff87e93 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:29.817471 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd704378-6a0a-42ac-9228-73b3aa054d39 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:29.830539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682432b9-1882-43aa-a6f1-32525e181f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:29.831538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682432b9-1882-43aa-a6f1-32525e181f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.831838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682432b9-1882-43aa-a6f1-32525e181f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.832171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682432b9-1882-43aa-a6f1-32525e181f33 admin admin] PATH_INFO: `/v3/projects/a5a0086895b6442da90bc0f1bcecd3e7/users/f7cf8ed5de4b41b19b6f25299087e1b5/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.878100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682432b9-1882-43aa-a6f1-32525e181f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a0086895b6442da90bc0f1bcecd3e7, user_id=f7cf8ed5de4b41b19b6f25299087e1b5, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:29.881290 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682432b9-1882-43aa-a6f1-32525e181f33 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:29.901723 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b658e9d1-70d8-4ea0-9475-1563dff87e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f114b074822e40f2bdb0a28b6adfc3da'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:29.903570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b658e9d1-70d8-4ea0-9475-1563dff87e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f114b074822e40f2bdb0a28b6adfc3da'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:29.905549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b658e9d1-70d8-4ea0-9475-1563dff87e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f114b074822e40f2bdb0a28b6adfc3da'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:29.908305 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 987/1904] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:29 2022] PUT /identity/v3/projects/a5a0086895b6442da90bc0f1bcecd3e7/users/f7cf8ed5de4b41b19b6f25299087e1b5/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:29.917784 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91eb14-8eda-4e9f-89da-dcafcf56031e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.918089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91eb14-8eda-4e9f-89da-dcafcf56031e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.918365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91eb14-8eda-4e9f-89da-dcafcf56031e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:29.920763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b658e9d1-70d8-4ea0-9475-1563dff87e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f114b074822e40f2bdb0a28b6adfc3da'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:29.931373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b658e9d1-70d8-4ea0-9475-1563dff87e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f114b074822e40f2bdb0a28b6adfc3da'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:29.952276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 918/1905] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:29 2022] DELETE /identity/v3/users/f114b074822e40f2bdb0a28b6adfc3da => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:29.954631 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-fd91eb14-8eda-4e9f-89da-dcafcf56031e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:29.956228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fd91eb14-8eda-4e9f-89da-dcafcf56031e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:29.957389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9cde5-5467-40ad-a397-520f3c449a85 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:29.957878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9cde5-5467-40ad-a397-520f3c449a85 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:29.958474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9cde5-5467-40ad-a397-520f3c449a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:30.000491 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-33e9cde5-5467-40ad-a397-520f3c449a85 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:30.002060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-33e9cde5-5467-40ad-a397-520f3c449a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:30.088637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:30.153640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 988/1906] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:30.197230 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 919/1907] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:30.200771 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91eb14-8eda-4e9f-89da-dcafcf56031e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:30.203948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fd91eb14-8eda-4e9f-89da-dcafcf56031e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:30.249978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9cde5-5467-40ad-a397-520f3c449a85 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:30.251852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-33e9cde5-5467-40ad-a397-520f3c449a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:30.332730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91eb14-8eda-4e9f-89da-dcafcf56031e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:30.337565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eb623-a484-46cc-8784-96081f3b7877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:30.338203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eb623-a484-46cc-8784-96081f3b7877 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:30.338598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eb623-a484-46cc-8784-96081f3b7877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:30.338982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8eb623-a484-46cc-8784-96081f3b7877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:30.350079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9cde5-5467-40ad-a397-520f3c449a85 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:30.352695 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfa92f-d082-42b4-bd71-460b1280d38f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:30.360217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfa92f-d082-42b4-bd71-460b1280d38f service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:30.360640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfa92f-d082-42b4-bd71-460b1280d38f service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:30.361150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcfa92f-d082-42b4-bd71-460b1280d38f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:30.366606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eb623-a484-46cc-8784-96081f3b7877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:30.372049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8eb623-a484-46cc-8784-96081f3b7877 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:30.382050 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcfa92f-d082-42b4-bd71-460b1280d38f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:30.383513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcfa92f-d082-42b4-bd71-460b1280d38f service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:30.403179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 989/1908] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:30.409807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 920/1909] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:30.672495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eb623-a484-46cc-8784-96081f3b7877 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:30.674478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3d8eb623-a484-46cc-8784-96081f3b7877 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:30.700133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8eb623-a484-46cc-8784-96081f3b7877 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:30.709067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:30.711102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:30.713522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:30.714282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] PATH_INFO: `/v3/projects/3b88aebfb7d34bc8b30f40080dc23d32` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:30.730295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b88aebfb7d34bc8b30f40080dc23d32)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:30.731268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:30.833263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b88aebfb7d34bc8b30f40080dc23d32', 'request_id': 'req-80d32ea0-ed9c-4780-9f72-451a36898fec'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:30.847897 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b88aebfb7d34bc8b30f40080dc23d32', 'request_id': 'req-80d32ea0-ed9c-4780-9f72-451a36898fec'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:30.849670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:30.858272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b88aebfb7d34bc8b30f40080dc23d32', 'request_id': 'req-80d32ea0-ed9c-4780-9f72-451a36898fec'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:30.885004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] The token cache is being invalidate because project 3b88aebfb7d34bc8b30f40080dc23d32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:30.886919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:30.888158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:30.891622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 990/1910] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:30 2022] DELETE /identity/v3/projects/3b88aebfb7d34bc8b30f40080dc23d32 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:30.896659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfa92f-d082-42b4-bd71-460b1280d38f service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:30.920550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcfa92f-d082-42b4-bd71-460b1280d38f service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:31.011043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:31.011945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:31.012234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:31.012536 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] PATH_INFO: `/v3/users/ec36ae80eb9a4ea9ab6866e7bb55510f` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:31.015044 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec36ae80eb9a4ea9ab6866e7bb55510f)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:31.017464 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:31.093451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec36ae80eb9a4ea9ab6866e7bb55510f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:31.094234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec36ae80eb9a4ea9ab6866e7bb55510f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:31.094852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec36ae80eb9a4ea9ab6866e7bb55510f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:31.111886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec36ae80eb9a4ea9ab6866e7bb55510f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:31.121523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec36ae80eb9a4ea9ab6866e7bb55510f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:31.144433 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 921/1911] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:30 2022] DELETE /identity/v3/users/ec36ae80eb9a4ea9ab6866e7bb55510f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:31.675944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:31.679295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:31.749799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:31.751030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:31.764612 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d32ea0-ed9c-4780-9f72-451a36898fec admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:31.789445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:31.903999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3265e3-1108-450d-a554-3d4d6ac56706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:31.905050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3265e3-1108-450d-a554-3d4d6ac56706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:31.905478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3265e3-1108-450d-a554-3d4d6ac56706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:31.905762 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3265e3-1108-450d-a554-3d4d6ac56706 admin admin] PATH_INFO: `/v3/projects/5e0a9ccdd1f34556a78af4735e9f4813` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:31.907628 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3265e3-1108-450d-a554-3d4d6ac56706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e0a9ccdd1f34556a78af4735e9f4813)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:31.909920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3265e3-1108-450d-a554-3d4d6ac56706 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:31.912121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a2bd1-f104-4f2d-beee-b3ec6164c585 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:31.920398 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:31.921648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:31.921983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:31.922273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] PATH_INFO: `/v3/projects/68dbb92c4e934c2f93f23a11e6f75e24` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:31.923950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68dbb92c4e934c2f93f23a11e6f75e24)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:31.927028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:32.003507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68dbb92c4e934c2f93f23a11e6f75e24', 'request_id': 'req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.014749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9e3265e3-1108-450d-a554-3d4d6ac56706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e0a9ccdd1f34556a78af4735e9f4813', 'request_id': 'req-9e3265e3-1108-450d-a554-3d4d6ac56706'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.018845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68dbb92c4e934c2f93f23a11e6f75e24', 'request_id': 'req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.025360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9e3265e3-1108-450d-a554-3d4d6ac56706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e0a9ccdd1f34556a78af4735e9f4813', 'request_id': 'req-9e3265e3-1108-450d-a554-3d4d6ac56706'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.033064 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9e3265e3-1108-450d-a554-3d4d6ac56706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e0a9ccdd1f34556a78af4735e9f4813', 'request_id': 'req-9e3265e3-1108-450d-a554-3d4d6ac56706'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.033830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68dbb92c4e934c2f93f23a11e6f75e24', 'request_id': 'req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.051699 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9e3265e3-1108-450d-a554-3d4d6ac56706 admin admin] The token cache is being invalidate because project 5e0a9ccdd1f34556a78af4735e9f4813 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:32.052286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9e3265e3-1108-450d-a554-3d4d6ac56706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.053320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9e3265e3-1108-450d-a554-3d4d6ac56706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.056124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 991/1912] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:31 2022] DELETE /identity/v3/projects/5e0a9ccdd1f34556a78af4735e9f4813 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:32.057003 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] The token cache is being invalidate because project 68dbb92c4e934c2f93f23a11e6f75e24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:32.057296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.058000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.059876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 922/1913] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:31 2022] DELETE /identity/v3/projects/68dbb92c4e934c2f93f23a11e6f75e24 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:32.061474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc1900-6659-4824-8c42-fe181408608f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:32.061834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc1900-6659-4824-8c42-fe181408608f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:32.062602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc1900-6659-4824-8c42-fe181408608f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:32.063609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:32.065248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:32.092028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7bcc1900-6659-4824-8c42-fe181408608f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:32.094713 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7bcc1900-6659-4824-8c42-fe181408608f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:32.174772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4563bf7a-38a1-4fe7-9c37-0c22bd414b21 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:32.181694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 992/1914] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:32.184378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:32.185605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:32.186195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:32.186617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] PATH_INFO: `/v3/users/63aaebcb56cf48dc8d50d6a0c16fb1eb` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:32.188945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] RBAC: Authorizing `identity:delete_user(user_id=63aaebcb56cf48dc8d50d6a0c16fb1eb)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:32.189259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeeb50-10f9-467a-91f1-c7595f6429c2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:32.189812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeeb50-10f9-467a-91f1-c7595f6429c2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:32.190205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeeb50-10f9-467a-91f1-c7595f6429c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:32.192906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:32.218482 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-caeeeb50-10f9-467a-91f1-c7595f6429c2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:32.219588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-caeeeb50-10f9-467a-91f1-c7595f6429c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:32.245880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 993/1915] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:32.245880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeeeb50-10f9-467a-91f1-c7595f6429c2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:32.252579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeeeb50-10f9-467a-91f1-c7595f6429c2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:32.265627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e1f3a-2c93-4436-bc19-6d7fa2d872e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:32.266465 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e1f3a-2c93-4436-bc19-6d7fa2d872e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:32.266823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e1f3a-2c93-4436-bc19-6d7fa2d872e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:32.267217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e1f3a-2c93-4436-bc19-6d7fa2d872e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:32.268147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e1f3a-2c93-4436-bc19-6d7fa2d872e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:32.268537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e1f3a-2c93-4436-bc19-6d7fa2d872e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:32.270263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e1f3a-2c93-4436-bc19-6d7fa2d872e6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:32.274244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:32.290482 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 994/1916] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:55:32.295819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e1f3a-2c93-4436-bc19-6d7fa2d872e6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:32.298099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63aaebcb56cf48dc8d50d6a0c16fb1eb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.299668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63aaebcb56cf48dc8d50d6a0c16fb1eb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.300865 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63aaebcb56cf48dc8d50d6a0c16fb1eb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.305839 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:32.311348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e1f3a-2c93-4436-bc19-6d7fa2d872e6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:32.320705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd8711-dac7-482a-a501-0d62e1a94930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:32.322075 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd8711-dac7-482a-a501-0d62e1a94930 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:32.322857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd8711-dac7-482a-a501-0d62e1a94930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:32.323201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd8711-dac7-482a-a501-0d62e1a94930 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:32.324008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd8711-dac7-482a-a501-0d62e1a94930 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:32.327495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd8711-dac7-482a-a501-0d62e1a94930 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:32.328763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63aaebcb56cf48dc8d50d6a0c16fb1eb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.337361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63aaebcb56cf48dc8d50d6a0c16fb1eb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:32.359038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 923/1917] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:32 2022] DELETE /identity/v3/users/63aaebcb56cf48dc8d50d6a0c16fb1eb => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:32.362436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 995/1918] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:32 2022] POST /identity/v3/projects => generated 379 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:32.367368 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:32.368930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:32.437993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f976a-8a6a-4e3d-89cb-c055eca65fac admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:32.504031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd4ebb-2472-4d57-a8ef-2cac6cacad08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:32.504750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd4ebb-2472-4d57-a8ef-2cac6cacad08 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:32.505052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd4ebb-2472-4d57-a8ef-2cac6cacad08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:32.505337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd4ebb-2472-4d57-a8ef-2cac6cacad08 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:32.506062 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd4ebb-2472-4d57-a8ef-2cac6cacad08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:32.508086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd4ebb-2472-4d57-a8ef-2cac6cacad08 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:32.528350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 924/1919] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:32 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:32.532392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd8711-dac7-482a-a501-0d62e1a94930 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:32.544133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd8711-dac7-482a-a501-0d62e1a94930 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:32.549187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6cf58-18ca-4532-84a0-9f13eadaaa76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:32.549985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6cf58-18ca-4532-84a0-9f13eadaaa76 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:32.550310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6cf58-18ca-4532-84a0-9f13eadaaa76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:32.550655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6cf58-18ca-4532-84a0-9f13eadaaa76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:32.551503 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6cf58-18ca-4532-84a0-9f13eadaaa76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:32.553436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6cf58-18ca-4532-84a0-9f13eadaaa76 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:32.570257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 996/1920] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:32.571815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd4ebb-2472-4d57-a8ef-2cac6cacad08 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:32.583302 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd4ebb-2472-4d57-a8ef-2cac6cacad08 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:32.588526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cbfa8-9488-47db-9b30-19600c5705f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:32.589459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cbfa8-9488-47db-9b30-19600c5705f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:32.589820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cbfa8-9488-47db-9b30-19600c5705f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:32.590110 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2cbfa8-9488-47db-9b30-19600c5705f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:32.591078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cbfa8-9488-47db-9b30-19600c5705f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:32.592816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2cbfa8-9488-47db-9b30-19600c5705f6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:32.606755 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 925/1921] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:32.608781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbe68e-4f7d-4cf3-aa8d-25f66ea7e2b4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:32.609532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbe68e-4f7d-4cf3-aa8d-25f66ea7e2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:32.610030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bbe68e-4f7d-4cf3-aa8d-25f66ea7e2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:32.610248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cbfa8-9488-47db-9b30-19600c5705f6 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:32.622795 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2cbfa8-9488-47db-9b30-19600c5705f6 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:32.627786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a00d7-3768-40c5-8733-a7854273dedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:32.631049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a00d7-3768-40c5-8733-a7854273dedf admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:32.631411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a00d7-3768-40c5-8733-a7854273dedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:32.631800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a00d7-3768-40c5-8733-a7854273dedf admin admin] PATH_INFO: `/v3/projects/7bf9f42b20ea4af787a9f06c33fe2975/users/49151b69a59c4100b4f23a74b0e2af40/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:32.674946 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-66bbe68e-4f7d-4cf3-aa8d-25f66ea7e2b4 None None] MFA Rules not processed for user `f7cf8ed5de4b41b19b6f25299087e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:32.676992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-66bbe68e-4f7d-4cf3-aa8d-25f66ea7e2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:32.797790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a00d7-3768-40c5-8733-a7854273dedf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf9f42b20ea4af787a9f06c33fe2975, user_id=49151b69a59c4100b4f23a74b0e2af40, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:32.800909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a00d7-3768-40c5-8733-a7854273dedf admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:32.816614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 997/1922] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:32.830802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbe68e-4f7d-4cf3-aa8d-25f66ea7e2b4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:32.833040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-66bbe68e-4f7d-4cf3-aa8d-25f66ea7e2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:32.833483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 926/1923] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:32 2022] PUT /identity/v3/projects/7bf9f42b20ea4af787a9f06c33fe2975/users/49151b69a59c4100b4f23a74b0e2af40/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:32.835649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2d3bf-a8f4-4890-9d4a-ba0a2f475286 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:32.836052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2d3bf-a8f4-4890-9d4a-ba0a2f475286 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:32.836428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a2d3bf-a8f4-4890-9d4a-ba0a2f475286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:32.870052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-50a2d3bf-a8f4-4890-9d4a-ba0a2f475286 None None] MFA Rules not processed for user `49151b69a59c4100b4f23a74b0e2af40`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:32.873249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-50a2d3bf-a8f4-4890-9d4a-ba0a2f475286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:32.899928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:32.923464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bbe68e-4f7d-4cf3-aa8d-25f66ea7e2b4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:32.966258 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 927/1924] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:32.967520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2d3bf-a8f4-4890-9d4a-ba0a2f475286 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:32.969135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-50a2d3bf-a8f4-4890-9d4a-ba0a2f475286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:33.000476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569b6ab-75b7-483e-b758-dcd4d2ef431d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:33.002253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569b6ab-75b7-483e-b758-dcd4d2ef431d service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.002723 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569b6ab-75b7-483e-b758-dcd4d2ef431d service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.003175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569b6ab-75b7-483e-b758-dcd4d2ef431d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.039191 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:33.046395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a2d3bf-a8f4-4890-9d4a-ba0a2f475286 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:33.065231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:33.066033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.066322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.066602 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d admin admin] PATH_INFO: `/v3/projects/a9a40e2e991645589ba74cf66112628a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.070391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a40e2e991645589ba74cf66112628a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:33.073996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:33.135133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569b6ab-75b7-483e-b758-dcd4d2ef431d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:33.140034 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569b6ab-75b7-483e-b758-dcd4d2ef431d service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:33.165213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 998/1925] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:33.166735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569b6ab-75b7-483e-b758-dcd4d2ef431d service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:33.189225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569b6ab-75b7-483e-b758-dcd4d2ef431d service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:33.189766 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a40e2e991645589ba74cf66112628a', 'request_id': 'req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:33.207548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a40e2e991645589ba74cf66112628a', 'request_id': 'req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:33.220466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a40e2e991645589ba74cf66112628a', 'request_id': 'req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:33.236381 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d admin admin] The token cache is being invalidate because project a9a40e2e991645589ba74cf66112628a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:33.236723 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:33.237555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bde8ff63-7fd1-4ad6-b39d-e53348ecd98d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:33.239958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 928/1926] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:32 2022] DELETE /identity/v3/projects/a9a40e2e991645589ba74cf66112628a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:33.246802 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ae722-d59d-4406-896f-1047e4d5c9e8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.247227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ae722-d59d-4406-896f-1047e4d5c9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.248600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ae722-d59d-4406-896f-1047e4d5c9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.279087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-de6ae722-d59d-4406-896f-1047e4d5c9e8 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:33.281217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-de6ae722-d59d-4406-896f-1047e4d5c9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:33.288585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be26ca-f051-4006-804e-42d534f28237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:33.289463 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be26ca-f051-4006-804e-42d534f28237 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.289831 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be26ca-f051-4006-804e-42d534f28237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.290233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be26ca-f051-4006-804e-42d534f28237 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.291262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be26ca-f051-4006-804e-42d534f28237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:33.292761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 929/1927] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:33.293603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be26ca-f051-4006-804e-42d534f28237 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:33.306449 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d95456-927e-4f76-89ab-f95b9b19fa2e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.306829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d95456-927e-4f76-89ab-f95b9b19fa2e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.307186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d95456-927e-4f76-89ab-f95b9b19fa2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.335789 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b6d95456-927e-4f76-89ab-f95b9b19fa2e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:33.339357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b6d95456-927e-4f76-89ab-f95b9b19fa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:33.359414 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 930/1928] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:33.360558 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d95456-927e-4f76-89ab-f95b9b19fa2e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:33.361599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 999/1929] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:33 2022] POST /identity/v3/projects => generated 379 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:33.371824 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d95456-927e-4f76-89ab-f95b9b19fa2e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:33.375494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be26ca-f051-4006-804e-42d534f28237 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:33.383975 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-42be26ca-f051-4006-804e-42d534f28237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:33.454792 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:33.571638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b72590-251b-41b8-90e5-03f99797e567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:33.572410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b72590-251b-41b8-90e5-03f99797e567 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.572727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b72590-251b-41b8-90e5-03f99797e567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.579661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b72590-251b-41b8-90e5-03f99797e567 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.580603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b72590-251b-41b8-90e5-03f99797e567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:33.580963 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b72590-251b-41b8-90e5-03f99797e567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:33.584857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be26ca-f051-4006-804e-42d534f28237 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:33.586667 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b72590-251b-41b8-90e5-03f99797e567 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:33.602256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f36c83-e015-4d0d-bd7f-f80b6466639f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:33.602628 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f36c83-e015-4d0d-bd7f-f80b6466639f admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.602904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f36c83-e015-4d0d-bd7f-f80b6466639f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.603188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f36c83-e015-4d0d-bd7f-f80b6466639f admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.603911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f36c83-e015-4d0d-bd7f-f80b6466639f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:33.606228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f36c83-e015-4d0d-bd7f-f80b6466639f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:33.622546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 931/1930] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:33.627331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b72590-251b-41b8-90e5-03f99797e567 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:33.639971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b72590-251b-41b8-90e5-03f99797e567 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:33.647227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44270952-d88a-4dbd-bb00-ad95b91dac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:33.648159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44270952-d88a-4dbd-bb00-ad95b91dac6a admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.648549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44270952-d88a-4dbd-bb00-ad95b91dac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.648959 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44270952-d88a-4dbd-bb00-ad95b91dac6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.649919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44270952-d88a-4dbd-bb00-ad95b91dac6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:33.652416 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44270952-d88a-4dbd-bb00-ad95b91dac6a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:33.681542 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1000/1931] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:33 2022] POST /identity/v3/users => generated 290 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:33.685960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f36c83-e015-4d0d-bd7f-f80b6466639f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:33.697041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 932/1932] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:33 2022] POST /identity/v3/projects => generated 381 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:33.698980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44270952-d88a-4dbd-bb00-ad95b91dac6a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:33.708157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f36c83-e015-4d0d-bd7f-f80b6466639f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:33.791580 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44270952-d88a-4dbd-bb00-ad95b91dac6a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:33.798750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8509af2-b5fb-4d2b-80b9-6924f953967f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:33.799039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8509af2-b5fb-4d2b-80b9-6924f953967f admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.799343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8509af2-b5fb-4d2b-80b9-6924f953967f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.799750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8509af2-b5fb-4d2b-80b9-6924f953967f admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.800790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8509af2-b5fb-4d2b-80b9-6924f953967f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:33.803986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8509af2-b5fb-4d2b-80b9-6924f953967f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:33.830486 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 933/1933] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:33 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:33.834535 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8509af2-b5fb-4d2b-80b9-6924f953967f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:33.842827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:33.851328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8509af2-b5fb-4d2b-80b9-6924f953967f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:33.867489 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b22d2-7b8f-44b5-8ae1-a3692355c6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:33.868574 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b22d2-7b8f-44b5-8ae1-a3692355c6da admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.868574 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b22d2-7b8f-44b5-8ae1-a3692355c6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.871432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b22d2-7b8f-44b5-8ae1-a3692355c6da admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.873881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b22d2-7b8f-44b5-8ae1-a3692355c6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:33.880325 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b22d2-7b8f-44b5-8ae1-a3692355c6da admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:33.902090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2098e48-0d45-4795-9636-0befad197786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:33.902670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2098e48-0d45-4795-9636-0befad197786 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.902964 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2098e48-0d45-4795-9636-0befad197786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.903250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2098e48-0d45-4795-9636-0befad197786 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.903996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2098e48-0d45-4795-9636-0befad197786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:33.910674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2098e48-0d45-4795-9636-0befad197786 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:33.918327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 934/1934] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:33.921043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b22d2-7b8f-44b5-8ae1-a3692355c6da admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:33.944831 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1001/1935] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:33.946208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2098e48-0d45-4795-9636-0befad197786 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:33.946807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b22d2-7b8f-44b5-8ae1-a3692355c6da admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:33.960040 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b102648-86d9-45fb-9fda-fc6b9b381134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:33.960752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b102648-86d9-45fb-9fda-fc6b9b381134 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.961048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b102648-86d9-45fb-9fda-fc6b9b381134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.961333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b102648-86d9-45fb-9fda-fc6b9b381134 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.962079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b102648-86d9-45fb-9fda-fc6b9b381134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:33.965231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b102648-86d9-45fb-9fda-fc6b9b381134 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:33.970526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2098e48-0d45-4795-9636-0befad197786 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:33.983652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331eb59-6810-4a1d-a159-10768fc08db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:33.984637 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331eb59-6810-4a1d-a159-10768fc08db8 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:33.984964 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331eb59-6810-4a1d-a159-10768fc08db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:33.985250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331eb59-6810-4a1d-a159-10768fc08db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:33.985989 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331eb59-6810-4a1d-a159-10768fc08db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:33.987650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1331eb59-6810-4a1d-a159-10768fc08db8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:33.989150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 935/1936] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:33.993529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b102648-86d9-45fb-9fda-fc6b9b381134 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.006071 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1002/1937] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:34.006260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b102648-86d9-45fb-9fda-fc6b9b381134 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.007823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331eb59-6810-4a1d-a159-10768fc08db8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.011282 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840c8717-9896-4906-b35e-76b9612460a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.012432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c8717-9896-4906-b35e-76b9612460a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.012806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c8717-9896-4906-b35e-76b9612460a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.013024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840c8717-9896-4906-b35e-76b9612460a8 admin admin] PATH_INFO: `/v3/projects/819a11e6369948f59e57d5f172b2d53a/users/ebae6f444a744aa897c0c5dbdcba7d6c/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.025017 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331eb59-6810-4a1d-a159-10768fc08db8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.030021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f7eb4-54be-4b1f-ad3f-d700a245546c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.031022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f7eb4-54be-4b1f-ad3f-d700a245546c admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.031433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f7eb4-54be-4b1f-ad3f-d700a245546c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.031816 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f7eb4-54be-4b1f-ad3f-d700a245546c admin admin] PATH_INFO: `/v3/projects/a6aa1d31a67644b4b3c895774426e37e/users/6580b5b63cfc4492b3f1095cad67dd0b/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.032556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c8717-9896-4906-b35e-76b9612460a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=819a11e6369948f59e57d5f172b2d53a, user_id=ebae6f444a744aa897c0c5dbdcba7d6c, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.036293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840c8717-9896-4906-b35e-76b9612460a8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.049170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f7eb4-54be-4b1f-ad3f-d700a245546c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6aa1d31a67644b4b3c895774426e37e, user_id=6580b5b63cfc4492b3f1095cad67dd0b, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.051810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f7eb4-54be-4b1f-ad3f-d700a245546c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.055197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 936/1938] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:33 2022] PUT /identity/v3/projects/819a11e6369948f59e57d5f172b2d53a/users/ebae6f444a744aa897c0c5dbdcba7d6c/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:34.065613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc897748-192e-4392-9718-50c88ab24e5b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.066008 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc897748-192e-4392-9718-50c88ab24e5b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.066359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc897748-192e-4392-9718-50c88ab24e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.073525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1003/1939] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:34 2022] PUT /identity/v3/projects/a6aa1d31a67644b4b3c895774426e37e/users/6580b5b63cfc4492b3f1095cad67dd0b/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:34.079342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30439fe-d985-4764-a35a-bf6f7ac2ba21 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.079878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30439fe-d985-4764-a35a-bf6f7ac2ba21 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.080180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30439fe-d985-4764-a35a-bf6f7ac2ba21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.097942 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-cc897748-192e-4392-9718-50c88ab24e5b None None] MFA Rules not processed for user `ebae6f444a744aa897c0c5dbdcba7d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:34.099191 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cc897748-192e-4392-9718-50c88ab24e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:34.112693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-d30439fe-d985-4764-a35a-bf6f7ac2ba21 None None] MFA Rules not processed for user `6580b5b63cfc4492b3f1095cad67dd0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:34.114233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d30439fe-d985-4764-a35a-bf6f7ac2ba21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:34.148609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:34.181000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 937/1940] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:34.185128 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc897748-192e-4392-9718-50c88ab24e5b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.202243 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1004/1941] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:34.207775 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30439fe-d985-4764-a35a-bf6f7ac2ba21 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.279610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc897748-192e-4392-9718-50c88ab24e5b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.284534 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d10d5-01a6-4ab5-9366-eef71df796fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.286668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d10d5-01a6-4ab5-9366-eef71df796fc admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.286668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d10d5-01a6-4ab5-9366-eef71df796fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.286668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d10d5-01a6-4ab5-9366-eef71df796fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.287272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d10d5-01a6-4ab5-9366-eef71df796fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.290431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18d10d5-01a6-4ab5-9366-eef71df796fc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.300173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30439fe-d985-4764-a35a-bf6f7ac2ba21 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.306549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63eb3a-f797-4091-bb60-9790ff13dcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.307451 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63eb3a-f797-4091-bb60-9790ff13dcd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.307763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63eb3a-f797-4091-bb60-9790ff13dcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.308042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb63eb3a-f797-4091-bb60-9790ff13dcd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.308829 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63eb3a-f797-4091-bb60-9790ff13dcd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.310937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb63eb3a-f797-4091-bb60-9790ff13dcd4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.321089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 938/1942] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:34 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:34.324666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d10d5-01a6-4ab5-9366-eef71df796fc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.340328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1005/1943] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:34 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:34.344339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63eb3a-f797-4091-bb60-9790ff13dcd4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.359325 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:34.394705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d10d5-01a6-4ab5-9366-eef71df796fc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.410732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb63eb3a-f797-4091-bb60-9790ff13dcd4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.416536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50e95b-bd10-41c9-a400-93ad89ad88dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.417266 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50e95b-bd10-41c9-a400-93ad89ad88dc admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.417556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50e95b-bd10-41c9-a400-93ad89ad88dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.417840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a50e95b-bd10-41c9-a400-93ad89ad88dc admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.418535 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50e95b-bd10-41c9-a400-93ad89ad88dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.420806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a50e95b-bd10-41c9-a400-93ad89ad88dc admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.443339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1006/1944] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:34 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:34.447379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50e95b-bd10-41c9-a400-93ad89ad88dc admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.460304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a50e95b-bd10-41c9-a400-93ad89ad88dc admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.463531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e18ff-76f5-4f19-acf1-4d4e9b753a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.464254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e18ff-76f5-4f19-acf1-4d4e9b753a77 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.464560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e18ff-76f5-4f19-acf1-4d4e9b753a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.464884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859e18ff-76f5-4f19-acf1-4d4e9b753a77 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.465605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e18ff-76f5-4f19-acf1-4d4e9b753a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.466273 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4baec-d445-4678-bc5c-1ac81756d471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.467046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4baec-d445-4678-bc5c-1ac81756d471 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.467431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4baec-d445-4678-bc5c-1ac81756d471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.467549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859e18ff-76f5-4f19-acf1-4d4e9b753a77 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.467882 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4baec-d445-4678-bc5c-1ac81756d471 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.468739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4baec-d445-4678-bc5c-1ac81756d471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.470509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e4baec-d445-4678-bc5c-1ac81756d471 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.483145 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1007/1945] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:34.487579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4baec-d445-4678-bc5c-1ac81756d471 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.493464 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 939/1946] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:34 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:34.495641 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e18ff-76f5-4f19-acf1-4d4e9b753a77 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.499445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e4baec-d445-4678-bc5c-1ac81756d471 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.501855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:34.507372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6ecff-aa1e-438f-b703-83fe843c6d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.508339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ecff-aa1e-438f-b703-83fe843c6d5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.508623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ecff-aa1e-438f-b703-83fe843c6d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.508934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6ecff-aa1e-438f-b703-83fe843c6d5e admin admin] PATH_INFO: `/v3/projects/4227ca6847ec4b2aba90a16df3d44863/users/f82d5a69f874442f975b92e268e135f9/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.510835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859e18ff-76f5-4f19-acf1-4d4e9b753a77 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.517289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cd62e-441f-41a2-985f-d6772d7f1169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.517992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cd62e-441f-41a2-985f-d6772d7f1169 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.518281 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cd62e-441f-41a2-985f-d6772d7f1169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.518563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cd62e-441f-41a2-985f-d6772d7f1169 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.519711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cd62e-441f-41a2-985f-d6772d7f1169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.521732 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cd62e-441f-41a2-985f-d6772d7f1169 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.527966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6ecff-aa1e-438f-b703-83fe843c6d5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4227ca6847ec4b2aba90a16df3d44863, user_id=f82d5a69f874442f975b92e268e135f9, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.530780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6ecff-aa1e-438f-b703-83fe843c6d5e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.534055 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 940/1947] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:34.538806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cd62e-441f-41a2-985f-d6772d7f1169 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.547570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1008/1948] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:34 2022] PUT /identity/v3/projects/4227ca6847ec4b2aba90a16df3d44863/users/f82d5a69f874442f975b92e268e135f9/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:34.551570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6ecff-aa1e-438f-b703-83fe843c6d5e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.552290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cd62e-441f-41a2-985f-d6772d7f1169 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.627209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6ecff-aa1e-438f-b703-83fe843c6d5e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.632115 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8607163-8bdc-433d-8f5c-268cc99657df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.633112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8607163-8bdc-433d-8f5c-268cc99657df admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.633465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8607163-8bdc-433d-8f5c-268cc99657df admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.633572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d3e07-072d-4efb-8718-8682406ce1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.633969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8607163-8bdc-433d-8f5c-268cc99657df admin admin] PATH_INFO: `/v3/projects/3a50c17cb6064399bc0db7f9a221dde5/users/8b46bb47ae3441ef8f531bd35a962c27/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.634205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d3e07-072d-4efb-8718-8682406ce1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.634496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d3e07-072d-4efb-8718-8682406ce1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.634783 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d3e07-072d-4efb-8718-8682406ce1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.635515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d3e07-072d-4efb-8718-8682406ce1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.637169 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d3e07-072d-4efb-8718-8682406ce1a9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.649263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1009/1949] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:34.651264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8607163-8bdc-433d-8f5c-268cc99657df admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a50c17cb6064399bc0db7f9a221dde5, user_id=8b46bb47ae3441ef8f531bd35a962c27, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.653567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d3e07-072d-4efb-8718-8682406ce1a9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.654629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8607163-8bdc-433d-8f5c-268cc99657df admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.665119 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d3e07-072d-4efb-8718-8682406ce1a9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.674473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 941/1950] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:34 2022] PUT /identity/v3/projects/3a50c17cb6064399bc0db7f9a221dde5/users/8b46bb47ae3441ef8f531bd35a962c27/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:34.679594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8607163-8bdc-433d-8f5c-268cc99657df admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.730077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a2077-0e31-4ed3-80b1-3eb372cc1c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.730673 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a2077-0e31-4ed3-80b1-3eb372cc1c76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.730774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a2077-0e31-4ed3-80b1-3eb372cc1c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.730970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a2077-0e31-4ed3-80b1-3eb372cc1c76 admin admin] PATH_INFO: `/v3/projects/4227ca6847ec4b2aba90a16df3d44863/users/f82d5a69f874442f975b92e268e135f9/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.733032 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a2077-0e31-4ed3-80b1-3eb372cc1c76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4227ca6847ec4b2aba90a16df3d44863, user_id=f82d5a69f874442f975b92e268e135f9, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.736038 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a2077-0e31-4ed3-80b1-3eb372cc1c76 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.742196 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:34.757044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1010/1951] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:34 2022] PUT /identity/v3/projects/4227ca6847ec4b2aba90a16df3d44863/users/f82d5a69f874442f975b92e268e135f9/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:34.760429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d2704-3723-464f-a437-ec4af9f97c52 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.760755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d2704-3723-464f-a437-ec4af9f97c52 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.761041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d2704-3723-464f-a437-ec4af9f97c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.764576 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8607163-8bdc-433d-8f5c-268cc99657df admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.791222 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7f4d2704-3723-464f-a437-ec4af9f97c52 None None] MFA Rules not processed for user `f82d5a69f874442f975b92e268e135f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:34.792565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7f4d2704-3723-464f-a437-ec4af9f97c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:34.834800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0074c-2ce1-494c-b50e-0cb146f94d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.836101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0074c-2ce1-494c-b50e-0cb146f94d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.836464 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0074c-2ce1-494c-b50e-0cb146f94d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.836955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f0074c-2ce1-494c-b50e-0cb146f94d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.837778 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0074c-2ce1-494c-b50e-0cb146f94d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.839496 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f0074c-2ce1-494c-b50e-0cb146f94d2b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.852775 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 942/1952] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:34 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:34.855191 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0074c-2ce1-494c-b50e-0cb146f94d2b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.866514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f0074c-2ce1-494c-b50e-0cb146f94d2b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:34.871923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40bde5-69d1-411d-b223-8a53585df3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:34.873498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40bde5-69d1-411d-b223-8a53585df3e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.873793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40bde5-69d1-411d-b223-8a53585df3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.874102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40bde5-69d1-411d-b223-8a53585df3e3 admin admin] PATH_INFO: `/v3/projects/3a50c17cb6064399bc0db7f9a221dde5/users/8b46bb47ae3441ef8f531bd35a962c27/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.876554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40bde5-69d1-411d-b223-8a53585df3e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a50c17cb6064399bc0db7f9a221dde5, user_id=8b46bb47ae3441ef8f531bd35a962c27, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:34.879690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40bde5-69d1-411d-b223-8a53585df3e3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:34.882394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1011/1953] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:34 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:34.902133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 943/1954] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:34 2022] PUT /identity/v3/projects/3a50c17cb6064399bc0db7f9a221dde5/users/8b46bb47ae3441ef8f531bd35a962c27/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:34.908865 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d2704-3723-464f-a437-ec4af9f97c52 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:34.910183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7f4d2704-3723-464f-a437-ec4af9f97c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:34.914193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29e192-61d3-4891-b0b2-181b3b2fe74d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:34.914193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29e192-61d3-4891-b0b2-181b3b2fe74d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:34.914193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29e192-61d3-4891-b0b2-181b3b2fe74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:34.922448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:34.959561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8d29e192-61d3-4891-b0b2-181b3b2fe74d None None] MFA Rules not processed for user `8b46bb47ae3441ef8f531bd35a962c27`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:34.961208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8d29e192-61d3-4891-b0b2-181b3b2fe74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:35.114438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d2704-3723-464f-a437-ec4af9f97c52 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:35.130282 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 944/1955] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:34 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:35.135699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fb152-09ac-4a79-bf0f-69f09a080645 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:35.135993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fb152-09ac-4a79-bf0f-69f09a080645 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:35.136281 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fb152-09ac-4a79-bf0f-69f09a080645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:35.170998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-088fb152-09ac-4a79-bf0f-69f09a080645 None None] MFA Rules not processed for user `f7cf8ed5de4b41b19b6f25299087e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:35.172939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-088fb152-09ac-4a79-bf0f-69f09a080645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:35.212044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a73de5-9ea2-46a8-b691-ffdcafd0f94b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:35.213241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a73de5-9ea2-46a8-b691-ffdcafd0f94b service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:35.213742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a73de5-9ea2-46a8-b691-ffdcafd0f94b service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:35.214244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a73de5-9ea2-46a8-b691-ffdcafd0f94b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:35.298967 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:35.312059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a73de5-9ea2-46a8-b691-ffdcafd0f94b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:35.313554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a73de5-9ea2-46a8-b691-ffdcafd0f94b service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:35.325963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1012/1956] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:35.326942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a73de5-9ea2-46a8-b691-ffdcafd0f94b service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:35.328464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-63a73de5-9ea2-46a8-b691-ffdcafd0f94b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:35.346960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a73de5-9ea2-46a8-b691-ffdcafd0f94b service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:35.363836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 945/1957] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:35.369435 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323af65-0bc6-4840-87a3-b4abbcff9a2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:35.370110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323af65-0bc6-4840-87a3-b4abbcff9a2b service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:35.370395 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323af65-0bc6-4840-87a3-b4abbcff9a2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:35.370676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323af65-0bc6-4840-87a3-b4abbcff9a2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:35.392276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323af65-0bc6-4840-87a3-b4abbcff9a2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:35.415041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323af65-0bc6-4840-87a3-b4abbcff9a2b service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:35.447272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1013/1958] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:35.488046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fb152-09ac-4a79-bf0f-69f09a080645 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:35.504625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fb152-09ac-4a79-bf0f-69f09a080645 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:35.527298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590e40-3cae-4979-b312-7e0776017f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:35.529245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590e40-3cae-4979-b312-7e0776017f73 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:35.530478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590e40-3cae-4979-b312-7e0776017f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:35.530934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39590e40-3cae-4979-b312-7e0776017f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:35.540111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-39590e40-3cae-4979-b312-7e0776017f73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:35.665237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323af65-0bc6-4840-87a3-b4abbcff9a2b service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:35.674988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323af65-0bc6-4840-87a3-b4abbcff9a2b service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:35.686446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce231f7-92fa-4647-aa65-bd7d9bde41fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:35.687310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce231f7-92fa-4647-aa65-bd7d9bde41fe service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:35.687955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce231f7-92fa-4647-aa65-bd7d9bde41fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:35.687955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce231f7-92fa-4647-aa65-bd7d9bde41fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:35.724411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39590e40-3cae-4979-b312-7e0776017f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:35.726719 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39590e40-3cae-4979-b312-7e0776017f73 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:35.745337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 946/1959] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:35.822285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce231f7-92fa-4647-aa65-bd7d9bde41fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:35.823446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce231f7-92fa-4647-aa65-bd7d9bde41fe service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:35.845633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1014/1960] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:38.533793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590e40-3cae-4979-b312-7e0776017f73 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:38.568398 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:38.604079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39590e40-3cae-4979-b312-7e0776017f73 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:38.611047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:38.612272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:38.612713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:38.613085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] PATH_INFO: `/v3/users/ebae6f444a744aa897c0c5dbdcba7d6c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:38.614582 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebae6f444a744aa897c0c5dbdcba7d6c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:38.617232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:38.705669 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebae6f444a744aa897c0c5dbdcba7d6c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:38.706490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebae6f444a744aa897c0c5dbdcba7d6c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:38.707192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebae6f444a744aa897c0c5dbdcba7d6c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:38.720255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebae6f444a744aa897c0c5dbdcba7d6c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:38.730524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebae6f444a744aa897c0c5dbdcba7d6c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:38.750037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 947/1961] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:38 2022] DELETE /identity/v3/users/ebae6f444a744aa897c0c5dbdcba7d6c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:38.754395 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092661a-72d7-400b-ad3d-783ad457c119 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:38.755395 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092661a-72d7-400b-ad3d-783ad457c119 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:38.755908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092661a-72d7-400b-ad3d-783ad457c119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:38.776148 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0092661a-72d7-400b-ad3d-783ad457c119 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:38.778849 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0092661a-72d7-400b-ad3d-783ad457c119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:38.869354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1015/1962] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:38.876597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:38.878087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:38.971665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092661a-72d7-400b-ad3d-783ad457c119 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:38.975729 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0092661a-72d7-400b-ad3d-783ad457c119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:38.981161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9838b-cd80-4388-8891-34fdd26e5787 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:38.985082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdedd8f-aa33-4999-b0cc-e965cb4ac7fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:38.985760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdedd8f-aa33-4999-b0cc-e965cb4ac7fd service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:38.986035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdedd8f-aa33-4999-b0cc-e965cb4ac7fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:38.986325 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdedd8f-aa33-4999-b0cc-e965cb4ac7fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:38.997556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdedd8f-aa33-4999-b0cc-e965cb4ac7fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:38.998886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092661a-72d7-400b-ad3d-783ad457c119 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:38.999213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdedd8f-aa33-4999-b0cc-e965cb4ac7fd service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:39.003990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:39.005057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:39.005446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:39.005843 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] PATH_INFO: `/v3/users/49151b69a59c4100b4f23a74b0e2af40` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:39.007849 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49151b69a59c4100b4f23a74b0e2af40)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:39.010060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:39.010914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 948/1963] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:39.072272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49151b69a59c4100b4f23a74b0e2af40'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.073639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49151b69a59c4100b4f23a74b0e2af40'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.075141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49151b69a59c4100b4f23a74b0e2af40'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.096269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49151b69a59c4100b4f23a74b0e2af40'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.099159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:39.104424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49151b69a59c4100b4f23a74b0e2af40'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.125154 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1016/1964] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:38 2022] DELETE /identity/v3/users/49151b69a59c4100b4f23a74b0e2af40 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:39.127800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb25c6-32f3-49b9-9635-cdba6d77e975 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:39.128256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb25c6-32f3-49b9-9635-cdba6d77e975 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:39.128750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eb25c6-32f3-49b9-9635-cdba6d77e975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:39.150059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-17eb25c6-32f3-49b9-9635-cdba6d77e975 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:39.152207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-17eb25c6-32f3-49b9-9635-cdba6d77e975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:39.225772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:39.249136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 949/1965] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:39.261347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:39.263069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:39.334158 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb25c6-32f3-49b9-9635-cdba6d77e975 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:39.336562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-17eb25c6-32f3-49b9-9635-cdba6d77e975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:39.356825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eb25c6-32f3-49b9-9635-cdba6d77e975 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:39.359490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74648e2-d224-45e1-8697-d0403ac5e4f9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:39.363056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:39.364124 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:39.364444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:39.364840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] PATH_INFO: `/v3/projects/819a11e6369948f59e57d5f172b2d53a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:39.365066 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ae7f7-37ef-4595-92fc-d208ce586685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:39.365876 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ae7f7-37ef-4595-92fc-d208ce586685 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:39.366328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ae7f7-37ef-4595-92fc-d208ce586685 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:39.366437 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=819a11e6369948f59e57d5f172b2d53a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:39.366741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ae7f7-37ef-4595-92fc-d208ce586685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:39.368739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:39.380323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ae7f7-37ef-4595-92fc-d208ce586685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:39.382549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ae7f7-37ef-4595-92fc-d208ce586685 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:39.394823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1017/1966] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:39.430210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819a11e6369948f59e57d5f172b2d53a', 'request_id': 'req-495beeef-84a6-40df-a009-3a574fb62949'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.441667 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819a11e6369948f59e57d5f172b2d53a', 'request_id': 'req-495beeef-84a6-40df-a009-3a574fb62949'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.451254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819a11e6369948f59e57d5f172b2d53a', 'request_id': 'req-495beeef-84a6-40df-a009-3a574fb62949'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.471855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] The token cache is being invalidate because project 819a11e6369948f59e57d5f172b2d53a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:39.472213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.473140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.475585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 950/1967] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:39 2022] DELETE /identity/v3/projects/819a11e6369948f59e57d5f172b2d53a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:39.479710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ae7f7-37ef-4595-92fc-d208ce586685 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:39.497698 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ae7f7-37ef-4595-92fc-d208ce586685 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:39.593434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:39.595361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:39.595361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:39.595361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] PATH_INFO: `/v3/users/8b46bb47ae3441ef8f531bd35a962c27` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:39.597127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b46bb47ae3441ef8f531bd35a962c27)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:39.599095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:39.688657 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b46bb47ae3441ef8f531bd35a962c27'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.689459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b46bb47ae3441ef8f531bd35a962c27'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.690374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b46bb47ae3441ef8f531bd35a962c27'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.720757 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b46bb47ae3441ef8f531bd35a962c27'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.731198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b46bb47ae3441ef8f531bd35a962c27'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:39.750072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1018/1968] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:39 2022] DELETE /identity/v3/users/8b46bb47ae3441ef8f531bd35a962c27 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:40.222867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:40.224945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:40.248329 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495beeef-84a6-40df-a009-3a574fb62949 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:40.364744 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:40.366046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:40.366516 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:40.367156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] PATH_INFO: `/v3/projects/7bf9f42b20ea4af787a9f06c33fe2975` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:40.370538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bf9f42b20ea4af787a9f06c33fe2975)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:40.373591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:40.477835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bf9f42b20ea4af787a9f06c33fe2975', 'request_id': 'req-69abeb58-75bb-4511-acbc-dcf767d08e41'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.494540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bf9f42b20ea4af787a9f06c33fe2975', 'request_id': 'req-69abeb58-75bb-4511-acbc-dcf767d08e41'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.508008 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bf9f42b20ea4af787a9f06c33fe2975', 'request_id': 'req-69abeb58-75bb-4511-acbc-dcf767d08e41'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.538804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] The token cache is being invalidate because project 7bf9f42b20ea4af787a9f06c33fe2975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:40.539161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.540421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.542692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 951/1969] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:40 2022] DELETE /identity/v3/projects/7bf9f42b20ea4af787a9f06c33fe2975 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:40.556491 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:40.560182 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:40.573878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:40.576398 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:40.586723 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07a9f8-855a-4e6d-85d4-eb86a45bf729 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:40.593865 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:40.601193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:40.681871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abeb58-75bb-4511-acbc-dcf767d08e41 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:40.688115 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55912473-2307-4361-821b-f96de12605c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:40.689246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55912473-2307-4361-821b-f96de12605c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:40.689701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55912473-2307-4361-821b-f96de12605c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:40.689985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55912473-2307-4361-821b-f96de12605c0 admin admin] PATH_INFO: `/v3/projects/3a50c17cb6064399bc0db7f9a221dde5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:40.691264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55912473-2307-4361-821b-f96de12605c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a50c17cb6064399bc0db7f9a221dde5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:40.693231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55912473-2307-4361-821b-f96de12605c0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:40.696436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dcdca9-6d7b-47fe-9edd-1931e766ffbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:40.697155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcdca9-6d7b-47fe-9edd-1931e766ffbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:40.697493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcdca9-6d7b-47fe-9edd-1931e766ffbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:40.697894 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dcdca9-6d7b-47fe-9edd-1931e766ffbf admin admin] PATH_INFO: `/v3/users/6580b5b63cfc4492b3f1095cad67dd0b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:40.699297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcdca9-6d7b-47fe-9edd-1931e766ffbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6580b5b63cfc4492b3f1095cad67dd0b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:40.701518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dcdca9-6d7b-47fe-9edd-1931e766ffbf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:40.755778 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-55912473-2307-4361-821b-f96de12605c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a50c17cb6064399bc0db7f9a221dde5', 'request_id': 'req-55912473-2307-4361-821b-f96de12605c0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.764083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-83dcdca9-6d7b-47fe-9edd-1931e766ffbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6580b5b63cfc4492b3f1095cad67dd0b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.764908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-83dcdca9-6d7b-47fe-9edd-1931e766ffbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6580b5b63cfc4492b3f1095cad67dd0b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.765586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-83dcdca9-6d7b-47fe-9edd-1931e766ffbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6580b5b63cfc4492b3f1095cad67dd0b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.766561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-55912473-2307-4361-821b-f96de12605c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a50c17cb6064399bc0db7f9a221dde5', 'request_id': 'req-55912473-2307-4361-821b-f96de12605c0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.775571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-55912473-2307-4361-821b-f96de12605c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a50c17cb6064399bc0db7f9a221dde5', 'request_id': 'req-55912473-2307-4361-821b-f96de12605c0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.779948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-83dcdca9-6d7b-47fe-9edd-1931e766ffbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6580b5b63cfc4492b3f1095cad67dd0b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.787668 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-83dcdca9-6d7b-47fe-9edd-1931e766ffbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6580b5b63cfc4492b3f1095cad67dd0b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.792699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-55912473-2307-4361-821b-f96de12605c0 admin admin] The token cache is being invalidate because project 3a50c17cb6064399bc0db7f9a221dde5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:40.793163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-55912473-2307-4361-821b-f96de12605c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.794593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-55912473-2307-4361-821b-f96de12605c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:40.800315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 952/1970] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:40 2022] DELETE /identity/v3/projects/3a50c17cb6064399bc0db7f9a221dde5 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:40.802552 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc02e4-8b97-420d-b6bb-7b2757031876 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:40.802864 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc02e4-8b97-420d-b6bb-7b2757031876 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:40.803237 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc02e4-8b97-420d-b6bb-7b2757031876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:40.817245 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1019/1971] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:40 2022] DELETE /identity/v3/users/6580b5b63cfc4492b3f1095cad67dd0b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:40.830402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-83dc02e4-8b97-420d-b6bb-7b2757031876 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:40.831639 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-83dc02e4-8b97-420d-b6bb-7b2757031876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:40.919463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 953/1972] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:40.928354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7eb54e-e04d-4c66-afc6-e4f316a57419 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:40.929339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7eb54e-e04d-4c66-afc6-e4f316a57419 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:40.929837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7eb54e-e04d-4c66-afc6-e4f316a57419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:40.957712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4e7eb54e-e04d-4c66-afc6-e4f316a57419 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:40.959330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4e7eb54e-e04d-4c66-afc6-e4f316a57419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:40.968759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1020/1973] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:40.973292 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc02e4-8b97-420d-b6bb-7b2757031876 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:40.990067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc02e4-8b97-420d-b6bb-7b2757031876 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:41.004082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34a7ab-353b-4e26-b9fa-c0f3bde2e104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:41.005213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34a7ab-353b-4e26-b9fa-c0f3bde2e104 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:41.005626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34a7ab-353b-4e26-b9fa-c0f3bde2e104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:41.005914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34a7ab-353b-4e26-b9fa-c0f3bde2e104 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:41.006723 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34a7ab-353b-4e26-b9fa-c0f3bde2e104 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:41.007855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34a7ab-353b-4e26-b9fa-c0f3bde2e104 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:41.009577 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34a7ab-353b-4e26-b9fa-c0f3bde2e104 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:41.025644 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 954/1974] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:41.030259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7eb54e-e04d-4c66-afc6-e4f316a57419 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:41.041225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7eb54e-e04d-4c66-afc6-e4f316a57419 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:41.048045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c11060-5285-4c8c-be1f-8eaafac57a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:41.049043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c11060-5285-4c8c-be1f-8eaafac57a45 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:41.049404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c11060-5285-4c8c-be1f-8eaafac57a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:41.049756 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c11060-5285-4c8c-be1f-8eaafac57a45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:41.050592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c11060-5285-4c8c-be1f-8eaafac57a45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:41.052747 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c11060-5285-4c8c-be1f-8eaafac57a45 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:41.109948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1021/1975] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:41 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:41.111268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34a7ab-353b-4e26-b9fa-c0f3bde2e104 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:41.124583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34a7ab-353b-4e26-b9fa-c0f3bde2e104 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:41.200645 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:41.260477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03977e08-c55e-478e-a318-7bf4298d9b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:41.261268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03977e08-c55e-478e-a318-7bf4298d9b74 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:41.261619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03977e08-c55e-478e-a318-7bf4298d9b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:41.261986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03977e08-c55e-478e-a318-7bf4298d9b74 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:41.262927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03977e08-c55e-478e-a318-7bf4298d9b74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:41.265017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03977e08-c55e-478e-a318-7bf4298d9b74 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:41.305335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 955/1976] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:41 2022] POST /identity/v3/users => generated 291 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:41.307705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c11060-5285-4c8c-be1f-8eaafac57a45 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:41.325561 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c11060-5285-4c8c-be1f-8eaafac57a45 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:41.340764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e9465-7b85-4536-91ae-2742e9733d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:41.341608 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e9465-7b85-4536-91ae-2742e9733d2e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:41.341903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e9465-7b85-4536-91ae-2742e9733d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:41.342187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e9465-7b85-4536-91ae-2742e9733d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:41.342926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e9465-7b85-4536-91ae-2742e9733d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:41.344554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e9465-7b85-4536-91ae-2742e9733d2e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:41.362511 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1022/1977] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:41.363568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03977e08-c55e-478e-a318-7bf4298d9b74 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:41.375984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03977e08-c55e-478e-a318-7bf4298d9b74 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:41.381441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61f767-e130-485a-aae0-8e9eb4ed139d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:41.382242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61f767-e130-485a-aae0-8e9eb4ed139d admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:41.382575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61f767-e130-485a-aae0-8e9eb4ed139d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:41.382930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61f767-e130-485a-aae0-8e9eb4ed139d admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:41.383976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61f767-e130-485a-aae0-8e9eb4ed139d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:41.386011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f61f767-e130-485a-aae0-8e9eb4ed139d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:41.408801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 956/1978] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:41.411383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e9465-7b85-4536-91ae-2742e9733d2e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:41.424940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e9465-7b85-4536-91ae-2742e9733d2e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:41.432914 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526cbdb-8f9d-4e18-b664-8fe0d982a2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:41.434735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526cbdb-8f9d-4e18-b664-8fe0d982a2be admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:41.435183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526cbdb-8f9d-4e18-b664-8fe0d982a2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:41.435609 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7526cbdb-8f9d-4e18-b664-8fe0d982a2be admin admin] PATH_INFO: `/v3/projects/716dfd5064be42b2b6c102fb4d6f353d/users/63df955d47a24eee8c036d86b3ad2ad9/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:41.458193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526cbdb-8f9d-4e18-b664-8fe0d982a2be admin admin] RBAC: Authorizing `identity:create_grant(project_id=716dfd5064be42b2b6c102fb4d6f353d, user_id=63df955d47a24eee8c036d86b3ad2ad9, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:41.461459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7526cbdb-8f9d-4e18-b664-8fe0d982a2be admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:41.483706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1023/1979] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:41 2022] PUT /identity/v3/projects/716dfd5064be42b2b6c102fb4d6f353d/users/63df955d47a24eee8c036d86b3ad2ad9/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:41.496727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49076a-bd33-4be5-9f99-9971973a5ce9 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:41.497529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49076a-bd33-4be5-9f99-9971973a5ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:41.498021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49076a-bd33-4be5-9f99-9971973a5ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:41.529985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526cbdb-8f9d-4e18-b664-8fe0d982a2be admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:41.533508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7526cbdb-8f9d-4e18-b664-8fe0d982a2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:41.598610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6b49076a-bd33-4be5-9f99-9971973a5ce9 None None] MFA Rules not processed for user `63df955d47a24eee8c036d86b3ad2ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:41.600270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6b49076a-bd33-4be5-9f99-9971973a5ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:41.637247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:41.733386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 957/1980] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:41.740622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49076a-bd33-4be5-9f99-9971973a5ce9 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:41.753895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7526cbdb-8f9d-4e18-b664-8fe0d982a2be admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:41.761734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:41.762912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:41.763613 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:41.764378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] PATH_INFO: `/v3/projects/a6aa1d31a67644b4b3c895774426e37e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:41.766189 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6aa1d31a67644b4b3c895774426e37e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:41.769001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:41.775787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49076a-bd33-4be5-9f99-9971973a5ce9 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:41.792480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125904eb-bdf5-44b8-9804-735a4365daa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:41.793399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125904eb-bdf5-44b8-9804-735a4365daa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:41.794406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125904eb-bdf5-44b8-9804-735a4365daa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:41.795787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125904eb-bdf5-44b8-9804-735a4365daa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:41.796617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125904eb-bdf5-44b8-9804-735a4365daa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:41.802746 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125904eb-bdf5-44b8-9804-735a4365daa5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:41.849679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6aa1d31a67644b4b3c895774426e37e', 'request_id': 'req-090baf2a-c1bc-4be3-8687-62f61ee6b905'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:41.872776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6aa1d31a67644b4b3c895774426e37e', 'request_id': 'req-090baf2a-c1bc-4be3-8687-62f61ee6b905'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:41.878698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 958/1981] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:41 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:41.879840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125904eb-bdf5-44b8-9804-735a4365daa5 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:41.884909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6aa1d31a67644b4b3c895774426e37e', 'request_id': 'req-090baf2a-c1bc-4be3-8687-62f61ee6b905'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:41.900454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] The token cache is being invalidate because project a6aa1d31a67644b4b3c895774426e37e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:41.901324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:41.902793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:41.906062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1024/1982] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:41 2022] DELETE /identity/v3/projects/a6aa1d31a67644b4b3c895774426e37e => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:41.909836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:41.911403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.010314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125904eb-bdf5-44b8-9804-735a4365daa5 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.015752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2130ac-b086-4bf3-ae1e-6463b3e22cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.016588 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2130ac-b086-4bf3-ae1e-6463b3e22cfc admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.016905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2130ac-b086-4bf3-ae1e-6463b3e22cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.017249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2130ac-b086-4bf3-ae1e-6463b3e22cfc admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.018017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2130ac-b086-4bf3-ae1e-6463b3e22cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.021816 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090baf2a-c1bc-4be3-8687-62f61ee6b905 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.023691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2130ac-b086-4bf3-ae1e-6463b3e22cfc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.029192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.030114 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.030407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.030823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] PATH_INFO: `/v3/users/f82d5a69f874442f975b92e268e135f9` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.032090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f82d5a69f874442f975b92e268e135f9)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.034420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.056180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 959/1983] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:41 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:42.060389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2130ac-b086-4bf3-ae1e-6463b3e22cfc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.068201 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fb2130ac-b086-4bf3-ae1e-6463b3e22cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.087648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:42.092323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2130ac-b086-4bf3-ae1e-6463b3e22cfc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.098537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b77a1-da8a-43bb-a20f-fa3275dbd649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.099350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b77a1-da8a-43bb-a20f-fa3275dbd649 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.100310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b77a1-da8a-43bb-a20f-fa3275dbd649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.100873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b77a1-da8a-43bb-a20f-fa3275dbd649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.102224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b77a1-da8a-43bb-a20f-fa3275dbd649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.108663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b77a1-da8a-43bb-a20f-fa3275dbd649 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.124323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 960/1984] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:42.125302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f82d5a69f874442f975b92e268e135f9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.126988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f82d5a69f874442f975b92e268e135f9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.128235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b77a1-da8a-43bb-a20f-fa3275dbd649 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.129018 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f82d5a69f874442f975b92e268e135f9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.133116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8b9b77a1-da8a-43bb-a20f-fa3275dbd649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.145234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:42.155259 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b77a1-da8a-43bb-a20f-fa3275dbd649 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.171208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f82d5a69f874442f975b92e268e135f9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.178773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3900d0-6d4c-4be0-a7b7-fc3be45fabfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.180317 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3900d0-6d4c-4be0-a7b7-fc3be45fabfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.180615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3900d0-6d4c-4be0-a7b7-fc3be45fabfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.180928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3900d0-6d4c-4be0-a7b7-fc3be45fabfd admin admin] PATH_INFO: `/v3/projects/6d12b4035e734c04969cfd070ba66bb2/users/720f508f203b401381dd39cedba724b3/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.181157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f82d5a69f874442f975b92e268e135f9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.204097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3900d0-6d4c-4be0-a7b7-fc3be45fabfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d12b4035e734c04969cfd070ba66bb2, user_id=720f508f203b401381dd39cedba724b3, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.206866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3900d0-6d4c-4be0-a7b7-fc3be45fabfd admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.207087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1025/1985] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:41 2022] DELETE /identity/v3/users/f82d5a69f874442f975b92e268e135f9 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:42.224974 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 961/1986] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:42 2022] PUT /identity/v3/projects/6d12b4035e734c04969cfd070ba66bb2/users/720f508f203b401381dd39cedba724b3/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:42.229002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.324232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf3e7e-6959-4a0c-962a-b4be97abcd67 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.391618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7249c0e-051c-47d3-a1ea-d7285eb5e2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.392354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7249c0e-051c-47d3-a1ea-d7285eb5e2ae admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.392928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7249c0e-051c-47d3-a1ea-d7285eb5e2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.393225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7249c0e-051c-47d3-a1ea-d7285eb5e2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.394008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7249c0e-051c-47d3-a1ea-d7285eb5e2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.395744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7249c0e-051c-47d3-a1ea-d7285eb5e2ae admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.408729 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1026/1987] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:42.413329 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3900d0-6d4c-4be0-a7b7-fc3be45fabfd admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.426749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3900d0-6d4c-4be0-a7b7-fc3be45fabfd admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.432772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6016610-659c-4b4a-b3d4-b8a899f16fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.433667 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6016610-659c-4b4a-b3d4-b8a899f16fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.433952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6016610-659c-4b4a-b3d4-b8a899f16fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.434235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6016610-659c-4b4a-b3d4-b8a899f16fbb admin admin] PATH_INFO: `/v3/projects/6d12b4035e734c04969cfd070ba66bb2/users/720f508f203b401381dd39cedba724b3/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.436768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6016610-659c-4b4a-b3d4-b8a899f16fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d12b4035e734c04969cfd070ba66bb2, user_id=720f508f203b401381dd39cedba724b3, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.439710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6016610-659c-4b4a-b3d4-b8a899f16fbb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.463491 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 962/1988] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:42 2022] PUT /identity/v3/projects/6d12b4035e734c04969cfd070ba66bb2/users/720f508f203b401381dd39cedba724b3/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:42.465655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92a873-9426-4618-b2d8-aec39ed07940 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.466114 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92a873-9426-4618-b2d8-aec39ed07940 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.466463 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92a873-9426-4618-b2d8-aec39ed07940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.497565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-aa92a873-9426-4618-b2d8-aec39ed07940 None None] MFA Rules not processed for user `720f508f203b401381dd39cedba724b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:42.499001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-aa92a873-9426-4618-b2d8-aec39ed07940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.562409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6016610-659c-4b4a-b3d4-b8a899f16fbb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.564287 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c6016610-659c-4b4a-b3d4-b8a899f16fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.598392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1027/1989] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:42.609421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92a873-9426-4618-b2d8-aec39ed07940 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.611142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-aa92a873-9426-4618-b2d8-aec39ed07940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.627216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:42.660868 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6016610-659c-4b4a-b3d4-b8a899f16fbb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.668303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.669735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.670265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.670885 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec admin admin] PATH_INFO: `/v3/projects/4227ca6847ec4b2aba90a16df3d44863` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.672584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=4227ca6847ec4b2aba90a16df3d44863)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.674760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.736928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4227ca6847ec4b2aba90a16df3d44863', 'request_id': 'req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.746189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4227ca6847ec4b2aba90a16df3d44863', 'request_id': 'req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.746387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92a873-9426-4618-b2d8-aec39ed07940 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.753953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4227ca6847ec4b2aba90a16df3d44863', 'request_id': 'req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.765887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec admin admin] The token cache is being invalidate because project 4227ca6847ec4b2aba90a16df3d44863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:42.766241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.766927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c58736d1-25d1-4171-b3cf-fbbb0aeb84ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:42.769736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 963/1990] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:42 2022] DELETE /identity/v3/projects/4227ca6847ec4b2aba90a16df3d44863 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:42.774957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02fb17-b6d1-4335-9a11-d7193de75f9c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.775260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02fb17-b6d1-4335-9a11-d7193de75f9c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.775550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b02fb17-b6d1-4335-9a11-d7193de75f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.794870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2b02fb17-b6d1-4335-9a11-d7193de75f9c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:42.796084 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2b02fb17-b6d1-4335-9a11-d7193de75f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.805563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25434b44-55de-45d0-96e4-9c79bd78786f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.806213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25434b44-55de-45d0-96e4-9c79bd78786f service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.806503 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25434b44-55de-45d0-96e4-9c79bd78786f service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.806793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25434b44-55de-45d0-96e4-9c79bd78786f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.808574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-25434b44-55de-45d0-96e4-9c79bd78786f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.840926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:42.866079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 964/1991] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:42.874383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee48fb0-3462-4208-abcb-76b029676ba2 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.874659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee48fb0-3462-4208-abcb-76b029676ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.874938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee48fb0-3462-4208-abcb-76b029676ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.886918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25434b44-55de-45d0-96e4-9c79bd78786f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.888958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25434b44-55de-45d0-96e4-9c79bd78786f service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.893435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6ee48fb0-3462-4208-abcb-76b029676ba2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:42.894774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6ee48fb0-3462-4208-abcb-76b029676ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:42.906359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 965/1992] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:42.910273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee48fb0-3462-4208-abcb-76b029676ba2 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.915014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1028/1993] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:42.927574 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee48fb0-3462-4208-abcb-76b029676ba2 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:42.940536 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ffe1c-7861-40ef-904b-ab818f923d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:42.941514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ffe1c-7861-40ef-904b-ab818f923d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:42.941859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ffe1c-7861-40ef-904b-ab818f923d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:42.942290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161ffe1c-7861-40ef-904b-ab818f923d05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:42.943065 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ffe1c-7861-40ef-904b-ab818f923d05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:42.943409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ffe1c-7861-40ef-904b-ab818f923d05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:42.947968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161ffe1c-7861-40ef-904b-ab818f923d05 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:42.964748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 966/1994] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:42.966385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25434b44-55de-45d0-96e4-9c79bd78786f service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:42.986170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25434b44-55de-45d0-96e4-9c79bd78786f service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.002728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffa487-e4d7-4154-85fd-8919bca8df1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.002728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffa487-e4d7-4154-85fd-8919bca8df1b admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.002728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffa487-e4d7-4154-85fd-8919bca8df1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.003120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffa487-e4d7-4154-85fd-8919bca8df1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.003922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffa487-e4d7-4154-85fd-8919bca8df1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.006351 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ffa487-e4d7-4154-85fd-8919bca8df1b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.042320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1029/1995] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:42 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:43.044434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ffe1c-7861-40ef-904b-ab818f923d05 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.067711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161ffe1c-7861-40ef-904b-ab818f923d05 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.134515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8218e-2b87-4259-8650-d8b87c36ebbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.135261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8218e-2b87-4259-8650-d8b87c36ebbc admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.135650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8218e-2b87-4259-8650-d8b87c36ebbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.136001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8218e-2b87-4259-8650-d8b87c36ebbc admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.136829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8218e-2b87-4259-8650-d8b87c36ebbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.139026 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8218e-2b87-4259-8650-d8b87c36ebbc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.168833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 967/1996] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:43 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:43.172988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffa487-e4d7-4154-85fd-8919bca8df1b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.184896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffa487-e4d7-4154-85fd-8919bca8df1b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.190459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575a9f7-eec7-444a-ba9c-8905cb9b088c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.191254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575a9f7-eec7-444a-ba9c-8905cb9b088c admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.191621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575a9f7-eec7-444a-ba9c-8905cb9b088c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.191980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575a9f7-eec7-444a-ba9c-8905cb9b088c admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.193089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575a9f7-eec7-444a-ba9c-8905cb9b088c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.195010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575a9f7-eec7-444a-ba9c-8905cb9b088c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.211299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1030/1997] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:43.217873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8218e-2b87-4259-8650-d8b87c36ebbc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.232356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8218e-2b87-4259-8650-d8b87c36ebbc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.242142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812461c2-737f-47f7-a50f-98a456fb54e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.242836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812461c2-737f-47f7-a50f-98a456fb54e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.243126 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812461c2-737f-47f7-a50f-98a456fb54e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.243410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812461c2-737f-47f7-a50f-98a456fb54e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.244153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812461c2-737f-47f7-a50f-98a456fb54e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.247049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812461c2-737f-47f7-a50f-98a456fb54e3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.273921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 968/1998] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:43.278348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575a9f7-eec7-444a-ba9c-8905cb9b088c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.290866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575a9f7-eec7-444a-ba9c-8905cb9b088c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.296778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0603fb-f34a-42ab-9b52-bb9ad77e6777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.306944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0603fb-f34a-42ab-9b52-bb9ad77e6777 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.307245 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0603fb-f34a-42ab-9b52-bb9ad77e6777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.307532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0603fb-f34a-42ab-9b52-bb9ad77e6777 admin admin] PATH_INFO: `/v3/projects/f8d4e7330b974d9c8d2583ad2cfd5377/users/aadea69428804891a6eee06289484962/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.323432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0603fb-f34a-42ab-9b52-bb9ad77e6777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8d4e7330b974d9c8d2583ad2cfd5377, user_id=aadea69428804891a6eee06289484962, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.326262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0603fb-f34a-42ab-9b52-bb9ad77e6777 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.340909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1031/1999] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:43 2022] PUT /identity/v3/projects/f8d4e7330b974d9c8d2583ad2cfd5377/users/aadea69428804891a6eee06289484962/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:43.353230 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81a654-a260-4df1-93ea-4b55141d5063 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.353596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81a654-a260-4df1-93ea-4b55141d5063 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.353969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81a654-a260-4df1-93ea-4b55141d5063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.382504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3e81a654-a260-4df1-93ea-4b55141d5063 None None] MFA Rules not processed for user `aadea69428804891a6eee06289484962`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:43.384034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3e81a654-a260-4df1-93ea-4b55141d5063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:43.429067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0603fb-f34a-42ab-9b52-bb9ad77e6777 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.430905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5b0603fb-f34a-42ab-9b52-bb9ad77e6777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:43.456141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:43.474823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 969/2000] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:43.480659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81a654-a260-4df1-93ea-4b55141d5063 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.517221 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:43.520080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0603fb-f34a-42ab-9b52-bb9ad77e6777 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.525961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bb9889-2623-45fc-97ed-8dabab02d411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.527206 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb9889-2623-45fc-97ed-8dabab02d411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.527496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb9889-2623-45fc-97ed-8dabab02d411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.527782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bb9889-2623-45fc-97ed-8dabab02d411 admin admin] PATH_INFO: `/v3/users/63df955d47a24eee8c036d86b3ad2ad9` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.529020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb9889-2623-45fc-97ed-8dabab02d411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63df955d47a24eee8c036d86b3ad2ad9)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.530944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bb9889-2623-45fc-97ed-8dabab02d411 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.549567 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81a654-a260-4df1-93ea-4b55141d5063 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.554320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6cc88-8c53-4fda-84cc-45ce0e90ef60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.554985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6cc88-8c53-4fda-84cc-45ce0e90ef60 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.555276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6cc88-8c53-4fda-84cc-45ce0e90ef60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.555554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6cc88-8c53-4fda-84cc-45ce0e90ef60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.556294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6cc88-8c53-4fda-84cc-45ce0e90ef60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.558402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6cc88-8c53-4fda-84cc-45ce0e90ef60 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.581061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-07bb9889-2623-45fc-97ed-8dabab02d411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63df955d47a24eee8c036d86b3ad2ad9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:43.581767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-07bb9889-2623-45fc-97ed-8dabab02d411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63df955d47a24eee8c036d86b3ad2ad9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:43.582403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-07bb9889-2623-45fc-97ed-8dabab02d411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63df955d47a24eee8c036d86b3ad2ad9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:43.585794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 970/2001] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:43 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:43.589788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6cc88-8c53-4fda-84cc-45ce0e90ef60 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.591689 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6bc6cc88-8c53-4fda-84cc-45ce0e90ef60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:43.598955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-07bb9889-2623-45fc-97ed-8dabab02d411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63df955d47a24eee8c036d86b3ad2ad9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:43.605790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-07bb9889-2623-45fc-97ed-8dabab02d411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63df955d47a24eee8c036d86b3ad2ad9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:43.619686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1032/2002] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:43 2022] DELETE /identity/v3/users/63df955d47a24eee8c036d86b3ad2ad9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:43.624530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20956e-2f77-4a57-951b-d6a5ca6a7c0d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.624854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20956e-2f77-4a57-951b-d6a5ca6a7c0d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.625577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20956e-2f77-4a57-951b-d6a5ca6a7c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.644867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ef20956e-2f77-4a57-951b-d6a5ca6a7c0d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:43.646104 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ef20956e-2f77-4a57-951b-d6a5ca6a7c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:43.659716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6cc88-8c53-4fda-84cc-45ce0e90ef60 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.707787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1033/2003] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:43.718033 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20956e-2f77-4a57-951b-d6a5ca6a7c0d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.719349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ef20956e-2f77-4a57-951b-d6a5ca6a7c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:43.721895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579f324-6dad-4bda-aaeb-1dc429c19f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.722579 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579f324-6dad-4bda-aaeb-1dc429c19f96 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.722871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579f324-6dad-4bda-aaeb-1dc429c19f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.723172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579f324-6dad-4bda-aaeb-1dc429c19f96 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.723840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579f324-6dad-4bda-aaeb-1dc429c19f96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.725838 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579f324-6dad-4bda-aaeb-1dc429c19f96 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.748168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 971/2004] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:43 2022] POST /identity/v3/users => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:43.752390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579f324-6dad-4bda-aaeb-1dc429c19f96 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.756452 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:43.763529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579f324-6dad-4bda-aaeb-1dc429c19f96 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.768741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e96954-3d8a-4d00-9e72-f670eec65234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.769421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e96954-3d8a-4d00-9e72-f670eec65234 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.769705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e96954-3d8a-4d00-9e72-f670eec65234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.769991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e96954-3d8a-4d00-9e72-f670eec65234 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.770739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e96954-3d8a-4d00-9e72-f670eec65234 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.772342 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e96954-3d8a-4d00-9e72-f670eec65234 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.783670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 972/2005] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:43.787455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20956e-2f77-4a57-951b-d6a5ca6a7c0d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.787810 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e96954-3d8a-4d00-9e72-f670eec65234 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.791371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc666385-eb41-42bd-8acc-6ba12f1d035b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.792020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc666385-eb41-42bd-8acc-6ba12f1d035b service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.792305 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc666385-eb41-42bd-8acc-6ba12f1d035b service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.792586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc666385-eb41-42bd-8acc-6ba12f1d035b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.799459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e96954-3d8a-4d00-9e72-f670eec65234 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.804907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc666385-eb41-42bd-8acc-6ba12f1d035b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.806387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc666385-eb41-42bd-8acc-6ba12f1d035b service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.806874 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fd343-87e3-4021-9652-2618682aa751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.807820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fd343-87e3-4021-9652-2618682aa751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.808112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fd343-87e3-4021-9652-2618682aa751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.808396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917fd343-87e3-4021-9652-2618682aa751 admin admin] PATH_INFO: `/v3/projects/2b150fc9340b4b93b536085399788128/users/c7ab3e4d000e469e9dc8c5c716b4291e/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.816899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1034/2006] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:43.840315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fd343-87e3-4021-9652-2618682aa751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b150fc9340b4b93b536085399788128, user_id=c7ab3e4d000e469e9dc8c5c716b4291e, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.843020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917fd343-87e3-4021-9652-2618682aa751 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.860463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 973/2007] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:43 2022] PUT /identity/v3/projects/2b150fc9340b4b93b536085399788128/users/c7ab3e4d000e469e9dc8c5c716b4291e/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:43.864805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc666385-eb41-42bd-8acc-6ba12f1d035b service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.877538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc666385-eb41-42bd-8acc-6ba12f1d035b service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.953832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3f815-f016-4eee-b0ed-cc62650d7d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.954618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3f815-f016-4eee-b0ed-cc62650d7d0d admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.954969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3f815-f016-4eee-b0ed-cc62650d7d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.955314 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3f815-f016-4eee-b0ed-cc62650d7d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.956145 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3f815-f016-4eee-b0ed-cc62650d7d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.957864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3f815-f016-4eee-b0ed-cc62650d7d0d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:43.968692 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1035/2008] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:43.973113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fd343-87e3-4021-9652-2618682aa751 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:43.983657 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917fd343-87e3-4021-9652-2618682aa751 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:43.988754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac4973-0792-426d-a9d9-42b9f281455e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:43.989721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac4973-0792-426d-a9d9-42b9f281455e admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:43.990078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac4973-0792-426d-a9d9-42b9f281455e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:43.990432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dac4973-0792-426d-a9d9-42b9f281455e admin admin] PATH_INFO: `/v3/projects/2b150fc9340b4b93b536085399788128/users/c7ab3e4d000e469e9dc8c5c716b4291e/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:43.993103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac4973-0792-426d-a9d9-42b9f281455e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b150fc9340b4b93b536085399788128, user_id=c7ab3e4d000e469e9dc8c5c716b4291e, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:43.995929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dac4973-0792-426d-a9d9-42b9f281455e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.018440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 974/2009] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:43 2022] PUT /identity/v3/projects/2b150fc9340b4b93b536085399788128/users/c7ab3e4d000e469e9dc8c5c716b4291e/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:44.021137 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ca07c-2e24-4df5-98ba-71ecb9c9e894 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.021430 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ca07c-2e24-4df5-98ba-71ecb9c9e894 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.021712 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ca07c-2e24-4df5-98ba-71ecb9c9e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.051599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7f1ca07c-2e24-4df5-98ba-71ecb9c9e894 None None] MFA Rules not processed for user `c7ab3e4d000e469e9dc8c5c716b4291e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:44.053223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7f1ca07c-2e24-4df5-98ba-71ecb9c9e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:44.151604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1036/2010] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:44.164596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac4973-0792-426d-a9d9-42b9f281455e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:44.166818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5dac4973-0792-426d-a9d9-42b9f281455e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:44.178259 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:44.237195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dac4973-0792-426d-a9d9-42b9f281455e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:44.240501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7198c6d-ba74-40b2-8a94-ec19d22f304a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:44.241208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7198c6d-ba74-40b2-8a94-ec19d22f304a service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.242039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7198c6d-ba74-40b2-8a94-ec19d22f304a service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.242565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7198c6d-ba74-40b2-8a94-ec19d22f304a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.245120 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e7198c6d-ba74-40b2-8a94-ec19d22f304a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:44.262364 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ca07c-2e24-4df5-98ba-71ecb9c9e894 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:44.263910 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7f1ca07c-2e24-4df5-98ba-71ecb9c9e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:44.334881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7198c6d-ba74-40b2-8a94-ec19d22f304a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:44.336385 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7198c6d-ba74-40b2-8a94-ec19d22f304a service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.347433 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 975/2011] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:44.392199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:44.398284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ca07c-2e24-4df5-98ba-71ecb9c9e894 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:44.403502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:44.404524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.404909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.405257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] PATH_INFO: `/v3/projects/716dfd5064be42b2b6c102fb4d6f353d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.406525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=716dfd5064be42b2b6c102fb4d6f353d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:44.408564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.464898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '716dfd5064be42b2b6c102fb4d6f353d', 'request_id': 'req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.476121 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '716dfd5064be42b2b6c102fb4d6f353d', 'request_id': 'req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.485973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '716dfd5064be42b2b6c102fb4d6f353d', 'request_id': 'req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.501807 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] The token cache is being invalidate because project 716dfd5064be42b2b6c102fb4d6f353d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:44.502115 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.502796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.507616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1037/2012] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:44 2022] DELETE /identity/v3/projects/716dfd5064be42b2b6c102fb4d6f353d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:44.509288 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7198c6d-ba74-40b2-8a94-ec19d22f304a service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:44.520808 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7198c6d-ba74-40b2-8a94-ec19d22f304a service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:44.611761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06fc2f-2636-43ce-b240-07fd50a44582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:44.613342 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06fc2f-2636-43ce-b240-07fd50a44582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:44.614454 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06fc2f-2636-43ce-b240-07fd50a44582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:44.614899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06fc2f-2636-43ce-b240-07fd50a44582 admin admin] PATH_INFO: `/v3/users/720f508f203b401381dd39cedba724b3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:44.617506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06fc2f-2636-43ce-b240-07fd50a44582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=720f508f203b401381dd39cedba724b3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:44.620694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06fc2f-2636-43ce-b240-07fd50a44582 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:44.721692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7b06fc2f-2636-43ce-b240-07fd50a44582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '720f508f203b401381dd39cedba724b3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.722834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7b06fc2f-2636-43ce-b240-07fd50a44582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '720f508f203b401381dd39cedba724b3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.724063 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7b06fc2f-2636-43ce-b240-07fd50a44582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '720f508f203b401381dd39cedba724b3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.740108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:44.741853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7b06fc2f-2636-43ce-b240-07fd50a44582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '720f508f203b401381dd39cedba724b3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.751783 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7b06fc2f-2636-43ce-b240-07fd50a44582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '720f508f203b401381dd39cedba724b3'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:44.770053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 976/2013] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:44 2022] DELETE /identity/v3/users/720f508f203b401381dd39cedba724b3 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:45.528545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:45.535844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:45.557402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a61252-1f9f-402d-b503-6ce9ba0b3e0f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:45.675686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84894c4-19a6-486b-8870-ffb1b345488d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:45.676737 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84894c4-19a6-486b-8870-ffb1b345488d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.677028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84894c4-19a6-486b-8870-ffb1b345488d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.677312 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84894c4-19a6-486b-8870-ffb1b345488d admin admin] PATH_INFO: `/v3/projects/6d12b4035e734c04969cfd070ba66bb2` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.678900 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84894c4-19a6-486b-8870-ffb1b345488d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d12b4035e734c04969cfd070ba66bb2)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:45.681081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84894c4-19a6-486b-8870-ffb1b345488d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:45.749413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c84894c4-19a6-486b-8870-ffb1b345488d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d12b4035e734c04969cfd070ba66bb2', 'request_id': 'req-c84894c4-19a6-486b-8870-ffb1b345488d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:45.769965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c84894c4-19a6-486b-8870-ffb1b345488d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d12b4035e734c04969cfd070ba66bb2', 'request_id': 'req-c84894c4-19a6-486b-8870-ffb1b345488d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:45.782148 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c84894c4-19a6-486b-8870-ffb1b345488d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d12b4035e734c04969cfd070ba66bb2', 'request_id': 'req-c84894c4-19a6-486b-8870-ffb1b345488d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:45.796038 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c84894c4-19a6-486b-8870-ffb1b345488d admin admin] The token cache is being invalidate because project 6d12b4035e734c04969cfd070ba66bb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:45.796873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c84894c4-19a6-486b-8870-ffb1b345488d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:45.797568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c84894c4-19a6-486b-8870-ffb1b345488d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:45.799365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1038/2014] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:45 2022] DELETE /identity/v3/projects/6d12b4035e734c04969cfd070ba66bb2 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:45.804453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472d826-cd3e-419e-a28c-353be0d194ee None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.805028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472d826-cd3e-419e-a28c-353be0d194ee None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.805128 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472d826-cd3e-419e-a28c-353be0d194ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.827648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e472d826-cd3e-419e-a28c-353be0d194ee None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:45.828095 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e472d826-cd3e-419e-a28c-353be0d194ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:45.896825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 977/2015] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:45.905493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d267a1a-4870-41a9-b992-21879b9e4ea8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.905955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d267a1a-4870-41a9-b992-21879b9e4ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.906340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d267a1a-4870-41a9-b992-21879b9e4ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.936340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6d267a1a-4870-41a9-b992-21879b9e4ea8 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:45.938496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6d267a1a-4870-41a9-b992-21879b9e4ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:45.951021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1039/2016] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:45.956131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472d826-cd3e-419e-a28c-353be0d194ee None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:45.971835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472d826-cd3e-419e-a28c-353be0d194ee None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:45.979646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b5a26-807f-4259-b433-5d79d1302ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:45.980355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b5a26-807f-4259-b433-5d79d1302ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:45.981430 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b5a26-807f-4259-b433-5d79d1302ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:45.981725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b5a26-807f-4259-b433-5d79d1302ad8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:45.982585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b5a26-807f-4259-b433-5d79d1302ad8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:45.983445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b5a26-807f-4259-b433-5d79d1302ad8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:45.997754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b5a26-807f-4259-b433-5d79d1302ad8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:46.012295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 978/2017] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:46.016282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d267a1a-4870-41a9-b992-21879b9e4ea8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:46.026545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d267a1a-4870-41a9-b992-21879b9e4ea8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:46.032326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d9afa-104f-4322-be08-a0bef54cbb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:46.033586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d9afa-104f-4322-be08-a0bef54cbb52 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:46.033884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d9afa-104f-4322-be08-a0bef54cbb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:46.034165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d9afa-104f-4322-be08-a0bef54cbb52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:46.034951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d9afa-104f-4322-be08-a0bef54cbb52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:46.036812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d9afa-104f-4322-be08-a0bef54cbb52 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:46.043507 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:46.079734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1040/2018] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:46 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:46.091551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b5a26-807f-4259-b433-5d79d1302ad8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:46.103733 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b5a26-807f-4259-b433-5d79d1302ad8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:46.229154 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b013cc9-091b-4f80-9bfb-f908cdb4a7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:46.230797 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b013cc9-091b-4f80-9bfb-f908cdb4a7fc admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:46.231083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b013cc9-091b-4f80-9bfb-f908cdb4a7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:46.231360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b013cc9-091b-4f80-9bfb-f908cdb4a7fc admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:46.232071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b013cc9-091b-4f80-9bfb-f908cdb4a7fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:46.234202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b013cc9-091b-4f80-9bfb-f908cdb4a7fc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:46.278507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 979/2019] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:46 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:46.279114 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d9afa-104f-4322-be08-a0bef54cbb52 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:46.312288 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d9afa-104f-4322-be08-a0bef54cbb52 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:46.336232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f9a79-abc9-48d7-9036-71d59e687f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:46.336966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f9a79-abc9-48d7-9036-71d59e687f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:46.337261 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f9a79-abc9-48d7-9036-71d59e687f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:46.337578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f9a79-abc9-48d7-9036-71d59e687f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:46.338437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f9a79-abc9-48d7-9036-71d59e687f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:46.354504 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f9a79-abc9-48d7-9036-71d59e687f05 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:46.391451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b013cc9-091b-4f80-9bfb-f908cdb4a7fc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:46.392048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1041/2020] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:46.408887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b013cc9-091b-4f80-9bfb-f908cdb4a7fc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:46.420993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233883e-d8ef-41db-82a3-a4509e05b0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:46.421688 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233883e-d8ef-41db-82a3-a4509e05b0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:46.421982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233883e-d8ef-41db-82a3-a4509e05b0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:46.422265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233883e-d8ef-41db-82a3-a4509e05b0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:46.423038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233883e-d8ef-41db-82a3-a4509e05b0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:46.424649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233883e-d8ef-41db-82a3-a4509e05b0c5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:46.448466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 980/2021] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:46.451811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f9a79-abc9-48d7-9036-71d59e687f05 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:46.483488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f9a79-abc9-48d7-9036-71d59e687f05 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:46.491999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee750ca-f081-4d4b-b6f0-45f9e7c0ea9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:46.494095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee750ca-f081-4d4b-b6f0-45f9e7c0ea9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:46.494517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee750ca-f081-4d4b-b6f0-45f9e7c0ea9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:46.494872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee750ca-f081-4d4b-b6f0-45f9e7c0ea9a admin admin] PATH_INFO: `/v3/projects/bcca37e950644ad6972d8183854e436a/users/fac8794d12e1423db9e738eca8c2469f/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:46.520952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee750ca-f081-4d4b-b6f0-45f9e7c0ea9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcca37e950644ad6972d8183854e436a, user_id=fac8794d12e1423db9e738eca8c2469f, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:46.526140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee750ca-f081-4d4b-b6f0-45f9e7c0ea9a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:46.542109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1042/2022] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:46 2022] PUT /identity/v3/projects/bcca37e950644ad6972d8183854e436a/users/fac8794d12e1423db9e738eca8c2469f/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:46.548043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda70338-0bb4-467f-8c0f-7842af2db22a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:46.549249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda70338-0bb4-467f-8c0f-7842af2db22a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:46.549548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda70338-0bb4-467f-8c0f-7842af2db22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:46.570290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-dda70338-0bb4-467f-8c0f-7842af2db22a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:46.571916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-dda70338-0bb4-467f-8c0f-7842af2db22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:46.634514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:46.681143 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 981/2023] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:46.703361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee750ca-f081-4d4b-b6f0-45f9e7c0ea9a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:46.705216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0ee750ca-f081-4d4b-b6f0-45f9e7c0ea9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:46.843103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee750ca-f081-4d4b-b6f0-45f9e7c0ea9a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:46.860482 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d131ba7-fde3-40ee-9ceb-e32a6ba8ea13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:46.861212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d131ba7-fde3-40ee-9ceb-e32a6ba8ea13 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:46.861503 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d131ba7-fde3-40ee-9ceb-e32a6ba8ea13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:46.861788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d131ba7-fde3-40ee-9ceb-e32a6ba8ea13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:46.880935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d131ba7-fde3-40ee-9ceb-e32a6ba8ea13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:46.883256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d131ba7-fde3-40ee-9ceb-e32a6ba8ea13 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:46.907773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1043/2024] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:48.205410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a970aeb-de54-4289-b8f0-182d62c3e02e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:48.205949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a970aeb-de54-4289-b8f0-182d62c3e02e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:48.206240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a970aeb-de54-4289-b8f0-182d62c3e02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:48.251534 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9a970aeb-de54-4289-b8f0-182d62c3e02e None None] MFA Rules not processed for user `fac8794d12e1423db9e738eca8c2469f`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:48.254070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9a970aeb-de54-4289-b8f0-182d62c3e02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:48.355697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 982/2025] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:48.356649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d131ba7-fde3-40ee-9ceb-e32a6ba8ea13 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:48.369876 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d131ba7-fde3-40ee-9ceb-e32a6ba8ea13 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:48.376909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f6e19-1242-416b-b9ee-548f526a46a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:48.377867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f6e19-1242-416b-b9ee-548f526a46a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:48.378447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f6e19-1242-416b-b9ee-548f526a46a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:48.378864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3f6e19-1242-416b-b9ee-548f526a46a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:48.379783 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f6e19-1242-416b-b9ee-548f526a46a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:48.382811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3f6e19-1242-416b-b9ee-548f526a46a0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:48.425523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1044/2026] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:48 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:48.430516 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a970aeb-de54-4289-b8f0-182d62c3e02e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:48.447432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a970aeb-de54-4289-b8f0-182d62c3e02e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:48.593041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c399d0-2085-4d91-a6cf-9ba0b6265a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:48.594232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c399d0-2085-4d91-a6cf-9ba0b6265a0c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:48.594673 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c399d0-2085-4d91-a6cf-9ba0b6265a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:48.595100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c399d0-2085-4d91-a6cf-9ba0b6265a0c admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:48.595964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c399d0-2085-4d91-a6cf-9ba0b6265a0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:48.598130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c399d0-2085-4d91-a6cf-9ba0b6265a0c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:48.631727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 983/2027] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:48 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:48.636285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f6e19-1242-416b-b9ee-548f526a46a0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:48.647338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3f6e19-1242-416b-b9ee-548f526a46a0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:48.652811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fae165-317a-4904-b02f-2a672baa3a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:48.653643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fae165-317a-4904-b02f-2a672baa3a0a admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:48.654028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fae165-317a-4904-b02f-2a672baa3a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:48.654575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fae165-317a-4904-b02f-2a672baa3a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:48.655431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fae165-317a-4904-b02f-2a672baa3a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:48.657301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fae165-317a-4904-b02f-2a672baa3a0a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:48.677341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1045/2028] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:48.679876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c399d0-2085-4d91-a6cf-9ba0b6265a0c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:48.692430 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c399d0-2085-4d91-a6cf-9ba0b6265a0c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:48.698997 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e2cd5-3ef2-4127-a133-f6cb0c212d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:48.700076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e2cd5-3ef2-4127-a133-f6cb0c212d83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:48.700357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e2cd5-3ef2-4127-a133-f6cb0c212d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:48.700636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e2cd5-3ef2-4127-a133-f6cb0c212d83 admin admin] PATH_INFO: `/v3/projects/8782d71800654541933c5358f49ef095/users/7b1ee165479140a3b2a2b456936f38c4/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:48.716028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e2cd5-3ef2-4127-a133-f6cb0c212d83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8782d71800654541933c5358f49ef095, user_id=7b1ee165479140a3b2a2b456936f38c4, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:48.718728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e2cd5-3ef2-4127-a133-f6cb0c212d83 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:48.732218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 984/2029] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:48 2022] PUT /identity/v3/projects/8782d71800654541933c5358f49ef095/users/7b1ee165479140a3b2a2b456936f38c4/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:48.736113 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fae165-317a-4904-b02f-2a672baa3a0a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:48.746811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fae165-317a-4904-b02f-2a672baa3a0a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:48.806143 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f501b-ec6d-48dd-805d-092b160251d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:48.806821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f501b-ec6d-48dd-805d-092b160251d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:48.807112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f501b-ec6d-48dd-805d-092b160251d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:48.807398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f501b-ec6d-48dd-805d-092b160251d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:48.808136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f501b-ec6d-48dd-805d-092b160251d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:48.809820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f501b-ec6d-48dd-805d-092b160251d1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:48.823045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1046/2030] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:48.824312 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e2cd5-3ef2-4127-a133-f6cb0c212d83 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:48.834696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e2cd5-3ef2-4127-a133-f6cb0c212d83 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:48.839545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7bb06-5d55-40f2-a7f3-522ab0b347ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:48.840456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7bb06-5d55-40f2-a7f3-522ab0b347ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:48.840782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7bb06-5d55-40f2-a7f3-522ab0b347ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:48.841058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7bb06-5d55-40f2-a7f3-522ab0b347ea admin admin] PATH_INFO: `/v3/projects/8782d71800654541933c5358f49ef095/users/7b1ee165479140a3b2a2b456936f38c4/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:48.843051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7bb06-5d55-40f2-a7f3-522ab0b347ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=8782d71800654541933c5358f49ef095, user_id=7b1ee165479140a3b2a2b456936f38c4, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:48.845745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7bb06-5d55-40f2-a7f3-522ab0b347ea admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:48.857333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 985/2031] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:48 2022] PUT /identity/v3/projects/8782d71800654541933c5358f49ef095/users/7b1ee165479140a3b2a2b456936f38c4/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:49.074690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f501b-ec6d-48dd-805d-092b160251d1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:49.079051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-853f501b-ec6d-48dd-805d-092b160251d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:49.112391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:49.195782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f501b-ec6d-48dd-805d-092b160251d1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:49.267087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db51ace-96c1-46f9-8400-7dd7c6c0fe26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:49.267942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db51ace-96c1-46f9-8400-7dd7c6c0fe26 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:49.268323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db51ace-96c1-46f9-8400-7dd7c6c0fe26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:49.268706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db51ace-96c1-46f9-8400-7dd7c6c0fe26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:49.271499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2db51ace-96c1-46f9-8400-7dd7c6c0fe26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:49.378351 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db51ace-96c1-46f9-8400-7dd7c6c0fe26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:49.380027 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db51ace-96c1-46f9-8400-7dd7c6c0fe26 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:49.403239 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1047/2032] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:50.072080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8617c9c-6e7a-4b1e-a5ca-37f6ea8ea6d6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:50.072791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8617c9c-6e7a-4b1e-a5ca-37f6ea8ea6d6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:50.072791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8617c9c-6e7a-4b1e-a5ca-37f6ea8ea6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:50.135016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f8617c9c-6e7a-4b1e-a5ca-37f6ea8ea6d6 None None] MFA Rules not processed for user `7b1ee165479140a3b2a2b456936f38c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:50.136324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f8617c9c-6e7a-4b1e-a5ca-37f6ea8ea6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:50.166022 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:50.243855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 986/2033] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:50 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:50.259786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db51ace-96c1-46f9-8400-7dd7c6c0fe26 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:50.269025 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db51ace-96c1-46f9-8400-7dd7c6c0fe26 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:50.273781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d54ac-ba08-4dc5-8ced-ed6fd7a4ee6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:50.274323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d54ac-ba08-4dc5-8ced-ed6fd7a4ee6b service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:50.274617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d54ac-ba08-4dc5-8ced-ed6fd7a4ee6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:50.274896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d54ac-ba08-4dc5-8ced-ed6fd7a4ee6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:50.345699 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d54ac-ba08-4dc5-8ced-ed6fd7a4ee6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:50.347069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d54ac-ba08-4dc5-8ced-ed6fd7a4ee6b service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:50.357717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1048/2034] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:51.726163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8617c9c-6e7a-4b1e-a5ca-37f6ea8ea6d6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:51.728459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f8617c9c-6e7a-4b1e-a5ca-37f6ea8ea6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:51.740381 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8617c9c-6e7a-4b1e-a5ca-37f6ea8ea6d6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:51.744647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1935b-2e19-43fb-ba99-852e4b43ee32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:51.745325 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1935b-2e19-43fb-ba99-852e4b43ee32 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:51.745610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1935b-2e19-43fb-ba99-852e4b43ee32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:51.745907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1935b-2e19-43fb-ba99-852e4b43ee32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:51.755568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1935b-2e19-43fb-ba99-852e4b43ee32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:51.757806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1935b-2e19-43fb-ba99-852e4b43ee32 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:51.767829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 987/2035] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:54.666849 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d54ac-ba08-4dc5-8ced-ed6fd7a4ee6b service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:54.668430 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2c8d54ac-ba08-4dc5-8ced-ed6fd7a4ee6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:54.715308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:54.773120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d54ac-ba08-4dc5-8ced-ed6fd7a4ee6b service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:54.779323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:54.780527 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:54.780702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:54.781146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] PATH_INFO: `/v3/users/aadea69428804891a6eee06289484962` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:54.787806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aadea69428804891a6eee06289484962)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:54.787806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:54.892915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aadea69428804891a6eee06289484962'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:54.894346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aadea69428804891a6eee06289484962'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:54.895104 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aadea69428804891a6eee06289484962'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:54.909791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aadea69428804891a6eee06289484962'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:54.919367 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aadea69428804891a6eee06289484962'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:54.956224 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1049/2036] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:54 2022] DELETE /identity/v3/users/aadea69428804891a6eee06289484962 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:54.960533 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad34fa6-ee00-40e5-872f-bddf30bd813e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:54.961041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad34fa6-ee00-40e5-872f-bddf30bd813e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:54.961132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad34fa6-ee00-40e5-872f-bddf30bd813e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:54.980785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-aad34fa6-ee00-40e5-872f-bddf30bd813e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:54.982513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-aad34fa6-ee00-40e5-872f-bddf30bd813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:55.046976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:55.048562 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:55.078886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 988/2037] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:55.089686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad34fa6-ee00-40e5-872f-bddf30bd813e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:55.091200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-aad34fa6-ee00-40e5-872f-bddf30bd813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:55.128824 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a1eff-d852-422a-ba47-7eaeb6dd7d81 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:55.135001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:55.136170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:55.136503 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:55.136950 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] PATH_INFO: `/v3/users/c3435419f72349b68c62a76638f98812` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:55.138336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3435419f72349b68c62a76638f98812)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:55.140856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:55.152573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad34fa6-ee00-40e5-872f-bddf30bd813e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:55.157786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611f460-4025-4f14-99ae-12b08cdc3521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:55.158529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611f460-4025-4f14-99ae-12b08cdc3521 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:55.158876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611f460-4025-4f14-99ae-12b08cdc3521 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:55.159333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1611f460-4025-4f14-99ae-12b08cdc3521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:55.171745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611f460-4025-4f14-99ae-12b08cdc3521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:55.173754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1611f460-4025-4f14-99ae-12b08cdc3521 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:55.189295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 989/2038] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:55.196596 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:55.199848 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3435419f72349b68c62a76638f98812'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.200889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3435419f72349b68c62a76638f98812'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.201674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3435419f72349b68c62a76638f98812'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.218113 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3435419f72349b68c62a76638f98812'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.226528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3435419f72349b68c62a76638f98812'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.244010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1050/2039] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:55 2022] DELETE /identity/v3/users/c3435419f72349b68c62a76638f98812 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:55.489772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611f460-4025-4f14-99ae-12b08cdc3521 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:55.499612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1611f460-4025-4f14-99ae-12b08cdc3521 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:55.540344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:55.542053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:55.567953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:55.585957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a760614-d58f-4b76-bdce-6862f39dea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:55.586906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a760614-d58f-4b76-bdce-6862f39dea2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:55.587315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a760614-d58f-4b76-bdce-6862f39dea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:55.587655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a760614-d58f-4b76-bdce-6862f39dea2a admin admin] PATH_INFO: `/v3/projects/6c041fd4759140b6926efa8c1a4935f7` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:55.588960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a760614-d58f-4b76-bdce-6862f39dea2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c041fd4759140b6926efa8c1a4935f7)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:55.590968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a760614-d58f-4b76-bdce-6862f39dea2a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:55.616975 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca9941-dac3-4c2f-878a-aa8f0d11ad53 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:55.622706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:55.623559 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:55.623847 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:55.624134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] PATH_INFO: `/v3/projects/f8d4e7330b974d9c8d2583ad2cfd5377` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:55.625389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8d4e7330b974d9c8d2583ad2cfd5377)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:55.627332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:55.639920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4a760614-d58f-4b76-bdce-6862f39dea2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c041fd4759140b6926efa8c1a4935f7', 'request_id': 'req-4a760614-d58f-4b76-bdce-6862f39dea2a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.648408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4a760614-d58f-4b76-bdce-6862f39dea2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c041fd4759140b6926efa8c1a4935f7', 'request_id': 'req-4a760614-d58f-4b76-bdce-6862f39dea2a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.656713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4a760614-d58f-4b76-bdce-6862f39dea2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c041fd4759140b6926efa8c1a4935f7', 'request_id': 'req-4a760614-d58f-4b76-bdce-6862f39dea2a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.671127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4a760614-d58f-4b76-bdce-6862f39dea2a admin admin] The token cache is being invalidate because project 6c041fd4759140b6926efa8c1a4935f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:55.671544 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4a760614-d58f-4b76-bdce-6862f39dea2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.672878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4a760614-d58f-4b76-bdce-6862f39dea2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.675475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 990/2040] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:55 2022] DELETE /identity/v3/projects/6c041fd4759140b6926efa8c1a4935f7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:55.681164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f70ff-eeb9-4e05-9054-f65ff5a21202 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:55.681455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f70ff-eeb9-4e05-9054-f65ff5a21202 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:55.681738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889f70ff-eeb9-4e05-9054-f65ff5a21202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:55.690293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8d4e7330b974d9c8d2583ad2cfd5377', 'request_id': 'req-b78e333d-3436-4639-a44b-9c97c8ae2448'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.698729 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8d4e7330b974d9c8d2583ad2cfd5377', 'request_id': 'req-b78e333d-3436-4639-a44b-9c97c8ae2448'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.703565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-889f70ff-eeb9-4e05-9054-f65ff5a21202 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:55.704954 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-889f70ff-eeb9-4e05-9054-f65ff5a21202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:55.706124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8d4e7330b974d9c8d2583ad2cfd5377', 'request_id': 'req-b78e333d-3436-4639-a44b-9c97c8ae2448'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.719278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] The token cache is being invalidate because project f8d4e7330b974d9c8d2583ad2cfd5377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:55.720076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.720585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.722636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1051/2041] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:55 2022] DELETE /identity/v3/projects/f8d4e7330b974d9c8d2583ad2cfd5377 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:55.726940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:55.728474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:55.787365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 991/2042] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:55.795915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4137-1f00-4a31-b2bf-afbc1b4c4f0e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:55.796555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4137-1f00-4a31-b2bf-afbc1b4c4f0e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:55.797335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4137-1f00-4a31-b2bf-afbc1b4c4f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:55.806110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:55.824338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78e333d-3436-4639-a44b-9c97c8ae2448 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:55.827743 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ceda4137-1f00-4a31-b2bf-afbc1b4c4f0e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:55.829154 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ceda4137-1f00-4a31-b2bf-afbc1b4c4f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:55.830263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:55.831354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:55.831752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:55.832187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] PATH_INFO: `/v3/users/c7ab3e4d000e469e9dc8c5c716b4291e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:55.833475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ab3e4d000e469e9dc8c5c716b4291e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:55.836090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:55.838327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 992/2043] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:55.842105 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda4137-1f00-4a31-b2bf-afbc1b4c4f0e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:55.852056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda4137-1f00-4a31-b2bf-afbc1b4c4f0e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:55.857068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63035f-c191-4107-bf8a-5401c05a4621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:55.857798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63035f-c191-4107-bf8a-5401c05a4621 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:55.858157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63035f-c191-4107-bf8a-5401c05a4621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:55.858506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63035f-c191-4107-bf8a-5401c05a4621 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:55.859308 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63035f-c191-4107-bf8a-5401c05a4621 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:55.859690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63035f-c191-4107-bf8a-5401c05a4621 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:55.861392 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63035f-c191-4107-bf8a-5401c05a4621 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:55.874860 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 993/2044] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:55:55.879080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63035f-c191-4107-bf8a-5401c05a4621 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:55.890464 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63035f-c191-4107-bf8a-5401c05a4621 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:55.895884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eca622-2636-4090-8d54-d08409128cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:55.896531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eca622-2636-4090-8d54-d08409128cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:55.896990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eca622-2636-4090-8d54-d08409128cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:55.897608 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ab3e4d000e469e9dc8c5c716b4291e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.898433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ab3e4d000e469e9dc8c5c716b4291e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.899130 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ab3e4d000e469e9dc8c5c716b4291e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.899828 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eca622-2636-4090-8d54-d08409128cc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:55.900560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eca622-2636-4090-8d54-d08409128cc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:55.902550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eca622-2636-4090-8d54-d08409128cc3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:55.914320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ab3e4d000e469e9dc8c5c716b4291e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.924415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ab3e4d000e469e9dc8c5c716b4291e'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:55.940821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1052/2045] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:55:55 2022] DELETE /identity/v3/users/c7ab3e4d000e469e9dc8c5c716b4291e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:55.946845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 994/2046] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:55 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:55.948048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:55.950462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:56.086840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c8305-657d-4680-bd1b-fb1f4232aae9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:56.253551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2b3a5-4a61-4f15-b627-31078230a539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:56.254286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2b3a5-4a61-4f15-b627-31078230a539 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.254576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2b3a5-4a61-4f15-b627-31078230a539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.254859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b2b3a5-4a61-4f15-b627-31078230a539 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.255873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b2b3a5-4a61-4f15-b627-31078230a539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:56.258648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b2b3a5-4a61-4f15-b627-31078230a539 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:56.289994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1053/2047] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:55 2022] POST /identity/v3/users => generated 291 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:56.293654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eca622-2636-4090-8d54-d08409128cc3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:56.306278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eca622-2636-4090-8d54-d08409128cc3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:56.311339 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423c13b-39db-4a0a-926e-16f7aabf60a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:56.312204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423c13b-39db-4a0a-926e-16f7aabf60a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.312498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423c13b-39db-4a0a-926e-16f7aabf60a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.312857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423c13b-39db-4a0a-926e-16f7aabf60a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.313805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423c13b-39db-4a0a-926e-16f7aabf60a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:56.315421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423c13b-39db-4a0a-926e-16f7aabf60a2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:56.337309 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 995/2048] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:56.337656 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2b3a5-4a61-4f15-b627-31078230a539 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:56.348268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b2b3a5-4a61-4f15-b627-31078230a539 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:56.354251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a56a50-a3f0-4af9-9f42-d2167ee5e27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:56.354919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a56a50-a3f0-4af9-9f42-d2167ee5e27c admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.355208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a56a50-a3f0-4af9-9f42-d2167ee5e27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.356022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a56a50-a3f0-4af9-9f42-d2167ee5e27c admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.356215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a56a50-a3f0-4af9-9f42-d2167ee5e27c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:56.357844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a56a50-a3f0-4af9-9f42-d2167ee5e27c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:56.359494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:56.374614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1054/2049] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:56.378766 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423c13b-39db-4a0a-926e-16f7aabf60a2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:56.390022 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423c13b-39db-4a0a-926e-16f7aabf60a2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:56.396890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc222f61-e9fe-4b2e-b362-ff77ea27a6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:56.397867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222f61-e9fe-4b2e-b362-ff77ea27a6db admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.398230 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222f61-e9fe-4b2e-b362-ff77ea27a6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.398585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc222f61-e9fe-4b2e-b362-ff77ea27a6db admin admin] PATH_INFO: `/v3/projects/2600d6dfa23a401d9d0283e37f1459bd/users/cfffc6d2733b4339aca0ad3291324f7b/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.418545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc222f61-e9fe-4b2e-b362-ff77ea27a6db admin admin] RBAC: Authorizing `identity:create_grant(project_id=2600d6dfa23a401d9d0283e37f1459bd, user_id=cfffc6d2733b4339aca0ad3291324f7b, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:56.421775 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc222f61-e9fe-4b2e-b362-ff77ea27a6db admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:56.433152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:56.438834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 996/2050] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:56 2022] PUT /identity/v3/projects/2600d6dfa23a401d9d0283e37f1459bd/users/cfffc6d2733b4339aca0ad3291324f7b/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:56.450637 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ee27b-f36d-4789-b689-18de767b5ce2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.450938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ee27b-f36d-4789-b689-18de767b5ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.451374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ee27b-f36d-4789-b689-18de767b5ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.454094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc222f61-e9fe-4b2e-b362-ff77ea27a6db admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:56.457162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bc222f61-e9fe-4b2e-b362-ff77ea27a6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:56.505709 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-290ee27b-f36d-4789-b689-18de767b5ce2 None None] MFA Rules not processed for user `cfffc6d2733b4339aca0ad3291324f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:56.507009 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-290ee27b-f36d-4789-b689-18de767b5ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:56.548736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc222f61-e9fe-4b2e-b362-ff77ea27a6db admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:56.553966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134dcdcd-d453-40e6-80b3-1d36d64971f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:56.556928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dcdcd-d453-40e6-80b3-1d36d64971f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.557240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dcdcd-d453-40e6-80b3-1d36d64971f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.557677 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134dcdcd-d453-40e6-80b3-1d36d64971f8 admin admin] PATH_INFO: `/v3/projects/2b150fc9340b4b93b536085399788128` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.560081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dcdcd-d453-40e6-80b3-1d36d64971f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b150fc9340b4b93b536085399788128)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:56.562046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134dcdcd-d453-40e6-80b3-1d36d64971f8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:56.588979 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1055/2051] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:56.591227 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ee27b-f36d-4789-b689-18de767b5ce2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:56.603250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ee27b-f36d-4789-b689-18de767b5ce2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:56.609458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213134a-b3a0-44f0-9216-ae654a5fcf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:56.610441 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213134a-b3a0-44f0-9216-ae654a5fcf05 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.611180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213134a-b3a0-44f0-9216-ae654a5fcf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.611657 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213134a-b3a0-44f0-9216-ae654a5fcf05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.612439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213134a-b3a0-44f0-9216-ae654a5fcf05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:56.615092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213134a-b3a0-44f0-9216-ae654a5fcf05 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:56.625562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-134dcdcd-d453-40e6-80b3-1d36d64971f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b150fc9340b4b93b536085399788128', 'request_id': 'req-134dcdcd-d453-40e6-80b3-1d36d64971f8'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:56.637350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-134dcdcd-d453-40e6-80b3-1d36d64971f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b150fc9340b4b93b536085399788128', 'request_id': 'req-134dcdcd-d453-40e6-80b3-1d36d64971f8'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:56.644842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-134dcdcd-d453-40e6-80b3-1d36d64971f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b150fc9340b4b93b536085399788128', 'request_id': 'req-134dcdcd-d453-40e6-80b3-1d36d64971f8'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:56.646452 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1056/2052] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:56 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:56.650329 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213134a-b3a0-44f0-9216-ae654a5fcf05 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:56.656826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-134dcdcd-d453-40e6-80b3-1d36d64971f8 admin admin] The token cache is being invalidate because project 2b150fc9340b4b93b536085399788128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:55:56.657121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-134dcdcd-d453-40e6-80b3-1d36d64971f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:56.657801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-134dcdcd-d453-40e6-80b3-1d36d64971f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:55:56.659386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 997/2053] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:55:56 2022] DELETE /identity/v3/projects/2b150fc9340b4b93b536085399788128 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:56.665948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ad094-0004-4e54-84b1-925e4669625f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.666351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ad094-0004-4e54-84b1-925e4669625f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.666736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ad094-0004-4e54-84b1-925e4669625f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.687321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-dc3ad094-0004-4e54-84b1-925e4669625f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:56.688555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-dc3ad094-0004-4e54-84b1-925e4669625f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:56.746614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213134a-b3a0-44f0-9216-ae654a5fcf05 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:56.752359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c499c7-580e-446f-996a-c42df9e6278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:56.753256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c499c7-580e-446f-996a-c42df9e6278b admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.753556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c499c7-580e-446f-996a-c42df9e6278b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.753834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c499c7-580e-446f-996a-c42df9e6278b admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.754632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c499c7-580e-446f-996a-c42df9e6278b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:56.756790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c499c7-580e-446f-996a-c42df9e6278b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:56.766239 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 998/2054] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:56.775891 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b86c6-5010-48f3-baab-6d759ffd64de None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.776186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b86c6-5010-48f3-baab-6d759ffd64de None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.776474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b86c6-5010-48f3-baab-6d759ffd64de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.795037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1057/2055] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:56 2022] POST /identity/v3/users => generated 296 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:56.796782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c499c7-580e-446f-996a-c42df9e6278b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:56.798538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b7c499c7-580e-446f-996a-c42df9e6278b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:56.801913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e42b86c6-5010-48f3-baab-6d759ffd64de None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:56.803803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e42b86c6-5010-48f3-baab-6d759ffd64de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:56.813317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c499c7-580e-446f-996a-c42df9e6278b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:56.819400 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b7ee-b4e6-4e90-94b0-bbd934759d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:56.820271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b7ee-b4e6-4e90-94b0-bbd934759d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.820568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b7ee-b4e6-4e90-94b0-bbd934759d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.820925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b7ee-b4e6-4e90-94b0-bbd934759d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.821669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43b7ee-b4e6-4e90-94b0-bbd934759d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:56.822921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 999/2056] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:56.823323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43b7ee-b4e6-4e90-94b0-bbd934759d76 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:56.827411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b86c6-5010-48f3-baab-6d759ffd64de None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:56.835858 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1058/2057] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:56.839724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b86c6-5010-48f3-baab-6d759ffd64de None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:56.840781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b7ee-b4e6-4e90-94b0-bbd934759d76 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:56.848475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb562f14-4825-40ec-a9e7-0bb098dab1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:56.849202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb562f14-4825-40ec-a9e7-0bb098dab1ed admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.849660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb562f14-4825-40ec-a9e7-0bb098dab1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.850083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb562f14-4825-40ec-a9e7-0bb098dab1ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.851408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb562f14-4825-40ec-a9e7-0bb098dab1ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:55:56.851724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb562f14-4825-40ec-a9e7-0bb098dab1ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:56.853165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b7ee-b4e6-4e90-94b0-bbd934759d76 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:56.853373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb562f14-4825-40ec-a9e7-0bb098dab1ed admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:56.864642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13039da-5023-4cdc-8cb2-7c7c1ffa164d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:56.864642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13039da-5023-4cdc-8cb2-7c7c1ffa164d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.864642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13039da-5023-4cdc-8cb2-7c7c1ffa164d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.864642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13039da-5023-4cdc-8cb2-7c7c1ffa164d admin admin] PATH_INFO: `/v3/projects/0860a96ad3844945b4a9960a7c5c4c87/users/be5997d6d7854f10931ab97c63fd76f1/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.871375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1000/2058] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:55:56.875906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb562f14-4825-40ec-a9e7-0bb098dab1ed admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:56.882414 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13039da-5023-4cdc-8cb2-7c7c1ffa164d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0860a96ad3844945b4a9960a7c5c4c87, user_id=be5997d6d7854f10931ab97c63fd76f1, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:56.886749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13039da-5023-4cdc-8cb2-7c7c1ffa164d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:56.887280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb562f14-4825-40ec-a9e7-0bb098dab1ed admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:56.892698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64038f51-a196-46c1-83c6-3c7ef4de194c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:56.893485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64038f51-a196-46c1-83c6-3c7ef4de194c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:56.893824 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64038f51-a196-46c1-83c6-3c7ef4de194c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:56.894175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64038f51-a196-46c1-83c6-3c7ef4de194c admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:56.895056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64038f51-a196-46c1-83c6-3c7ef4de194c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:56.897260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64038f51-a196-46c1-83c6-3c7ef4de194c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:56.902520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:56.904513 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1059/2059] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:56 2022] PUT /identity/v3/projects/0860a96ad3844945b4a9960a7c5c4c87/users/be5997d6d7854f10931ab97c63fd76f1/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:56.909175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13039da-5023-4cdc-8cb2-7c7c1ffa164d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:56.928192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1001/2060] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:55:56 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:55:56.931848 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:56.933716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64038f51-a196-46c1-83c6-3c7ef4de194c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:56.977390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13039da-5023-4cdc-8cb2-7c7c1ffa164d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:56.997088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64038f51-a196-46c1-83c6-3c7ef4de194c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:57.001831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaeffe2-bfc1-4da6-9aa8-81628cb1f3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:57.002507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaeffe2-bfc1-4da6-9aa8-81628cb1f3cb admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:57.002790 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaeffe2-bfc1-4da6-9aa8-81628cb1f3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:57.003066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaeffe2-bfc1-4da6-9aa8-81628cb1f3cb admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:57.003752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaeffe2-bfc1-4da6-9aa8-81628cb1f3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:57.005779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaeffe2-bfc1-4da6-9aa8-81628cb1f3cb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:57.039413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1002/2061] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:55:56 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:55:57.039605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c20a48-dd27-4fc2-afb9-d1b2bc7b6b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:57.039605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c20a48-dd27-4fc2-afb9-d1b2bc7b6b01 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:57.039605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c20a48-dd27-4fc2-afb9-d1b2bc7b6b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:57.039605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c20a48-dd27-4fc2-afb9-d1b2bc7b6b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:57.039910 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c20a48-dd27-4fc2-afb9-d1b2bc7b6b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:57.040469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaeffe2-bfc1-4da6-9aa8-81628cb1f3cb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:57.041122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c20a48-dd27-4fc2-afb9-d1b2bc7b6b01 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:57.050546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaeffe2-bfc1-4da6-9aa8-81628cb1f3cb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:57.052064 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1060/2062] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:57.056737 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c20a48-dd27-4fc2-afb9-d1b2bc7b6b01 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:57.057462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fda8d-068e-4443-942f-9cced9271346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:57.058023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fda8d-068e-4443-942f-9cced9271346 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:57.058224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fda8d-068e-4443-942f-9cced9271346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:57.058495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fda8d-068e-4443-942f-9cced9271346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:57.059226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fda8d-068e-4443-942f-9cced9271346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:57.060844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fda8d-068e-4443-942f-9cced9271346 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:57.068388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c20a48-dd27-4fc2-afb9-d1b2bc7b6b01 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:57.072064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1003/2063] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:55:57.075708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fda8d-068e-4443-942f-9cced9271346 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:57.076647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5c861-c147-4400-81d2-351dc4a69230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:57.077649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5c861-c147-4400-81d2-351dc4a69230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:57.078013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5c861-c147-4400-81d2-351dc4a69230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:57.078364 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5c861-c147-4400-81d2-351dc4a69230 admin admin] PATH_INFO: `/v3/projects/0860a96ad3844945b4a9960a7c5c4c87/users/be5997d6d7854f10931ab97c63fd76f1/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:57.080574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5c861-c147-4400-81d2-351dc4a69230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0860a96ad3844945b4a9960a7c5c4c87, user_id=be5997d6d7854f10931ab97c63fd76f1, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:57.083308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5c861-c147-4400-81d2-351dc4a69230 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:57.086905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fda8d-068e-4443-942f-9cced9271346 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:57.092153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c40d9b-16dd-402e-8bad-e94d5b78a6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:57.092887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c40d9b-16dd-402e-8bad-e94d5b78a6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:57.093185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c40d9b-16dd-402e-8bad-e94d5b78a6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:57.093560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c40d9b-16dd-402e-8bad-e94d5b78a6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:57.094247 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c40d9b-16dd-402e-8bad-e94d5b78a6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:57.096513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c40d9b-16dd-402e-8bad-e94d5b78a6d3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:57.098361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1061/2064] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:57 2022] PUT /identity/v3/projects/0860a96ad3844945b4a9960a7c5c4c87/users/be5997d6d7854f10931ab97c63fd76f1/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:55:57.110304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f9417-8a51-4fb7-aec6-5b61c734c2db None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:57.110604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f9417-8a51-4fb7-aec6-5b61c734c2db None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:57.110889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f9417-8a51-4fb7-aec6-5b61c734c2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:57.112664 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1004/2065] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:55:57.118129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c40d9b-16dd-402e-8bad-e94d5b78a6d3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:57.135021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c40d9b-16dd-402e-8bad-e94d5b78a6d3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:57.141536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c06f9417-8a51-4fb7-aec6-5b61c734c2db None None] MFA Rules not processed for user `be5997d6d7854f10931ab97c63fd76f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:57.143641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c06f9417-8a51-4fb7-aec6-5b61c734c2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:57.222703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d289016-5276-4763-ac10-44cedd022574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:57.223601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d289016-5276-4763-ac10-44cedd022574 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:57.223882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d289016-5276-4763-ac10-44cedd022574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:57.224379 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d289016-5276-4763-ac10-44cedd022574 admin admin] PATH_INFO: `/v3/projects/d4f7063c9ea64b569c3699a45ad0bb2e/users/23a9ad25030045b9b5dad89eadca1b9c/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:57.225051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1062/2066] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:55:57.237198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f9417-8a51-4fb7-aec6-5b61c734c2db None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:57.238755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c06f9417-8a51-4fb7-aec6-5b61c734c2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:57.249011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d289016-5276-4763-ac10-44cedd022574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4f7063c9ea64b569c3699a45ad0bb2e, user_id=23a9ad25030045b9b5dad89eadca1b9c, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:57.251685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d289016-5276-4763-ac10-44cedd022574 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:57.269337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1005/2067] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:55:57 2022] PUT /identity/v3/projects/d4f7063c9ea64b569c3699a45ad0bb2e/users/23a9ad25030045b9b5dad89eadca1b9c/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:55:57.274788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a8a82-d8e4-4f06-a3f3-67f4dfb5bb25 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:57.275391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a8a82-d8e4-4f06-a3f3-67f4dfb5bb25 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:57.275724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a8a82-d8e4-4f06-a3f3-67f4dfb5bb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:57.284697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:57.305057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a06a8a82-d8e4-4f06-a3f3-67f4dfb5bb25 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:57.306237 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a06a8a82-d8e4-4f06-a3f3-67f4dfb5bb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:57.325232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f9417-8a51-4fb7-aec6-5b61c734c2db None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:57.360703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:55:57.402417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e8978-f97a-4823-94d8-183ca634c465 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:57.404839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e8978-f97a-4823-94d8-183ca634c465 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:57.405240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e8978-f97a-4823-94d8-183ca634c465 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:57.405656 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e8978-f97a-4823-94d8-183ca634c465 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:57.406456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1006/2068] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:57.421545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a8a82-d8e4-4f06-a3f3-67f4dfb5bb25 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:57.422299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a06a8a82-d8e4-4f06-a3f3-67f4dfb5bb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:57.518111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e8978-f97a-4823-94d8-183ca634c465 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:57.520436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e8978-f97a-4823-94d8-183ca634c465 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:57.530101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a8a82-d8e4-4f06-a3f3-67f4dfb5bb25 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:57.533931 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8c56d-fa58-45ad-95a1-d5a93a900a71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:57.534596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8c56d-fa58-45ad-95a1-d5a93a900a71 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:57.534884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8c56d-fa58-45ad-95a1-d5a93a900a71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:57.535159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8c56d-fa58-45ad-95a1-d5a93a900a71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:57.540416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1063/2069] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:57.546431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8c56d-fa58-45ad-95a1-d5a93a900a71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:57.547944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8c56d-fa58-45ad-95a1-d5a93a900a71 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:57.559393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1007/2070] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:57.629599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e8978-f97a-4823-94d8-183ca634c465 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:57.639047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e8978-f97a-4823-94d8-183ca634c465 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:57.642947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158095c-f813-4e4f-a05a-4bf3085fb708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:57.643605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158095c-f813-4e4f-a05a-4bf3085fb708 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:57.643915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158095c-f813-4e4f-a05a-4bf3085fb708 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:57.644226 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158095c-f813-4e4f-a05a-4bf3085fb708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:57.646090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e158095c-f813-4e4f-a05a-4bf3085fb708 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:57.721220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158095c-f813-4e4f-a05a-4bf3085fb708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:57.723360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158095c-f813-4e4f-a05a-4bf3085fb708 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:57.743648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1064/2071] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:55:58.262682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677865e7-2a7e-4b1c-b32f-f253ee82c0e7 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:58.262974 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677865e7-2a7e-4b1c-b32f-f253ee82c0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:58.263257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677865e7-2a7e-4b1c-b32f-f253ee82c0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:58.293405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-677865e7-2a7e-4b1c-b32f-f253ee82c0e7 None None] MFA Rules not processed for user `23a9ad25030045b9b5dad89eadca1b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:58.294912 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-677865e7-2a7e-4b1c-b32f-f253ee82c0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:58.362132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1008/2072] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:55:58.373976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158095c-f813-4e4f-a05a-4bf3085fb708 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:55:58.383952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158095c-f813-4e4f-a05a-4bf3085fb708 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:55:58.387657 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e315150-74c9-491a-ab3a-50b2979f8472 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:55:58.388341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e315150-74c9-491a-ab3a-50b2979f8472 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:58.388633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e315150-74c9-491a-ab3a-50b2979f8472 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:58.388947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e315150-74c9-491a-ab3a-50b2979f8472 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:58.398505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e315150-74c9-491a-ab3a-50b2979f8472 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:55:58.399947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e315150-74c9-491a-ab3a-50b2979f8472 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:55:58.409155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1065/2073] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:55:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:55:59.793526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0274c2-377f-4573-b384-fb20ce414a9d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:55:59.794028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0274c2-377f-4573-b384-fb20ce414a9d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:55:59.794387 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0274c2-377f-4573-b384-fb20ce414a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:55:59.812507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8a0274c2-377f-4573-b384-fb20ce414a9d None None] MFA Rules not processed for user `23a9ad25030045b9b5dad89eadca1b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:55:59.813812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8a0274c2-377f-4573-b384-fb20ce414a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:55:59.821443 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1009/2074] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:55:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:07.844574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e315150-74c9-491a-ab3a-50b2979f8472 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:07.846113 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6e315150-74c9-491a-ab3a-50b2979f8472 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:07.860864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e315150-74c9-491a-ab3a-50b2979f8472 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:07.865416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:07.866828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:07.867121 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:07.867403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] PATH_INFO: `/v3/users/fac8794d12e1423db9e738eca8c2469f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:07.868602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac8794d12e1423db9e738eca8c2469f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:07.870578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:07.892149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:07.917035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac8794d12e1423db9e738eca8c2469f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:07.918088 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac8794d12e1423db9e738eca8c2469f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:07.919187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac8794d12e1423db9e738eca8c2469f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:07.933806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac8794d12e1423db9e738eca8c2469f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:07.940488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac8794d12e1423db9e738eca8c2469f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:07.952592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1066/2075] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:07 2022] DELETE /identity/v3/users/fac8794d12e1423db9e738eca8c2469f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:08.141639 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0274c2-377f-4573-b384-fb20ce414a9d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:08.155559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0274c2-377f-4573-b384-fb20ce414a9d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:08.169721 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:08.171818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:08.228722 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:08.229591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:08.229878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:08.230180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] PATH_INFO: `/v3/projects/bcca37e950644ad6972d8183854e436a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:08.231413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcca37e950644ad6972d8183854e436a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:08.233391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:08.240253 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:08.240879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d42d1-da14-436b-9dd6-b66ce17e19cb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:08.245959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad60b19-3c11-4a28-ad37-694429a5e9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:08.246857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad60b19-3c11-4a28-ad37-694429a5e9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:08.247139 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad60b19-3c11-4a28-ad37-694429a5e9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:08.247417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad60b19-3c11-4a28-ad37-694429a5e9e3 admin admin] PATH_INFO: `/v3/users/cfffc6d2733b4339aca0ad3291324f7b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:08.248621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad60b19-3c11-4a28-ad37-694429a5e9e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfffc6d2733b4339aca0ad3291324f7b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:08.250575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad60b19-3c11-4a28-ad37-694429a5e9e3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:08.290781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcca37e950644ad6972d8183854e436a', 'request_id': 'req-1de1e810-5abb-4060-bf41-18989b2d0c36'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.299234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcca37e950644ad6972d8183854e436a', 'request_id': 'req-1de1e810-5abb-4060-bf41-18989b2d0c36'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.307146 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcca37e950644ad6972d8183854e436a', 'request_id': 'req-1de1e810-5abb-4060-bf41-18989b2d0c36'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.308792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7ad60b19-3c11-4a28-ad37-694429a5e9e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfffc6d2733b4339aca0ad3291324f7b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.310197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7ad60b19-3c11-4a28-ad37-694429a5e9e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfffc6d2733b4339aca0ad3291324f7b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.310959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7ad60b19-3c11-4a28-ad37-694429a5e9e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfffc6d2733b4339aca0ad3291324f7b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.320358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] The token cache is being invalidate because project bcca37e950644ad6972d8183854e436a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:08.320786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.321521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.323377 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1010/2076] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:08 2022] DELETE /identity/v3/projects/bcca37e950644ad6972d8183854e436a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:08.326812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7ad60b19-3c11-4a28-ad37-694429a5e9e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfffc6d2733b4339aca0ad3291324f7b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.326991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:08.328544 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:08.339690 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7ad60b19-3c11-4a28-ad37-694429a5e9e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfffc6d2733b4339aca0ad3291324f7b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.351386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1067/2077] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:08 2022] DELETE /identity/v3/users/cfffc6d2733b4339aca0ad3291324f7b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:56:08.355978 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8c885-e2a6-46d3-ba8d-6521d3c98b40 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:08.356276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8c885-e2a6-46d3-ba8d-6521d3c98b40 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:08.356567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8c885-e2a6-46d3-ba8d-6521d3c98b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:08.374782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8dc8c885-e2a6-46d3-ba8d-6521d3c98b40 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:08.376041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8dc8c885-e2a6-46d3-ba8d-6521d3c98b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:08.387235 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:08.410834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1e810-5abb-4060-bf41-18989b2d0c36 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:08.448347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1068/2078] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:08.467616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8c885-e2a6-46d3-ba8d-6521d3c98b40 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:08.469392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8dc8c885-e2a6-46d3-ba8d-6521d3c98b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:08.486822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:08.487969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:08.488520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:08.489006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] PATH_INFO: `/v3/users/7b1ee165479140a3b2a2b456936f38c4` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:08.491164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b1ee165479140a3b2a2b456936f38c4)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:08.494706 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:08.539467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8c885-e2a6-46d3-ba8d-6521d3c98b40 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:08.544553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934439-4e77-4876-8a86-f45df2f114ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:08.545240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934439-4e77-4876-8a86-f45df2f114ac service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:08.545652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934439-4e77-4876-8a86-f45df2f114ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:08.545938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59934439-4e77-4876-8a86-f45df2f114ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:08.548412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:08.550837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b1ee165479140a3b2a2b456936f38c4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.551576 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b1ee165479140a3b2a2b456936f38c4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.552544 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b1ee165479140a3b2a2b456936f38c4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.562218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59934439-4e77-4876-8a86-f45df2f114ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:08.563954 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59934439-4e77-4876-8a86-f45df2f114ac service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:08.566348 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b1ee165479140a3b2a2b456936f38c4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.574328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b1ee165479140a3b2a2b456936f38c4'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.580785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1069/2079] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:56:08.598660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1011/2080] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:08 2022] DELETE /identity/v3/users/7b1ee165479140a3b2a2b456936f38c4 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:08.801368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934439-4e77-4876-8a86-f45df2f114ac service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:08.813223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59934439-4e77-4876-8a86-f45df2f114ac service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:08.890322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:08.891382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:08.891673 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:08.891958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0 admin admin] PATH_INFO: `/v3/projects/8782d71800654541933c5358f49ef095` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:08.893226 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8782d71800654541933c5358f49ef095)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:08.895141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:08.947004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8782d71800654541933c5358f49ef095', 'request_id': 'req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.954670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8782d71800654541933c5358f49ef095', 'request_id': 'req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.960370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:08.962066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:08.972236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:08.976422 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8782d71800654541933c5358f49ef095', 'request_id': 'req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.978338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c2c77-93b8-415c-a67a-a1675fc5127f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:08.993900 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0 admin admin] The token cache is being invalidate because project 8782d71800654541933c5358f49ef095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:08.994196 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.994963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dca76f9f-1fa3-4027-bf98-ca1f32edf7a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:08.996510 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1070/2081] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:08 2022] DELETE /identity/v3/projects/8782d71800654541933c5358f49ef095 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:09.002177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b08e26-5876-4395-93a0-bb9d57dffe94 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.002859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b08e26-5876-4395-93a0-bb9d57dffe94 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.003169 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b08e26-5876-4395-93a0-bb9d57dffe94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.022708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-83b08e26-5876-4395-93a0-bb9d57dffe94 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:09.023933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-83b08e26-5876-4395-93a0-bb9d57dffe94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:09.042232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:09.043136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.043429 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.043903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] PATH_INFO: `/v3/projects/2600d6dfa23a401d9d0283e37f1459bd` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.045335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2600d6dfa23a401d9d0283e37f1459bd)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:09.047272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:09.085024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1071/2082] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:56:09.093968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01ef8a-8861-48eb-a57e-785d21dac59a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.094557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01ef8a-8861-48eb-a57e-785d21dac59a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.094841 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d01ef8a-8861-48eb-a57e-785d21dac59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.103180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2600d6dfa23a401d9d0283e37f1459bd', 'request_id': 'req-8177657c-a4b9-47d0-b2b3-9a867a9d778f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.111562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2600d6dfa23a401d9d0283e37f1459bd', 'request_id': 'req-8177657c-a4b9-47d0-b2b3-9a867a9d778f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.114734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6d01ef8a-8861-48eb-a57e-785d21dac59a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:09.116538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6d01ef8a-8861-48eb-a57e-785d21dac59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:09.119767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2600d6dfa23a401d9d0283e37f1459bd', 'request_id': 'req-8177657c-a4b9-47d0-b2b3-9a867a9d778f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.130953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] The token cache is being invalidate because project 2600d6dfa23a401d9d0283e37f1459bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:09.131246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.131995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.133703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1012/2083] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:08 2022] DELETE /identity/v3/projects/2600d6dfa23a401d9d0283e37f1459bd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:56:09.137795 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:09.139338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:09.176700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1072/2084] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:09.181523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01ef8a-8861-48eb-a57e-785d21dac59a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:09.190721 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d01ef8a-8861-48eb-a57e-785d21dac59a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:09.193071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:09.196311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf4330-6c47-451b-8f95-83fd698f82af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:09.197159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf4330-6c47-451b-8f95-83fd698f82af admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.197664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf4330-6c47-451b-8f95-83fd698f82af admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.198251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf4330-6c47-451b-8f95-83fd698f82af admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.199216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf4330-6c47-451b-8f95-83fd698f82af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:56:09.199764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf4330-6c47-451b-8f95-83fd698f82af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:09.202053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf4330-6c47-451b-8f95-83fd698f82af admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:09.207915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8177657c-a4b9-47d0-b2b3-9a867a9d778f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:09.214221 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:09.218724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.218724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.218724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] PATH_INFO: `/v3/users/be5997d6d7854f10931ab97c63fd76f1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.218724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be5997d6d7854f10931ab97c63fd76f1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:09.219354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1073/2085] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:56:09.220767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:09.220976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf4330-6c47-451b-8f95-83fd698f82af admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:09.232710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf4330-6c47-451b-8f95-83fd698f82af admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:09.238522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8a0f8-4738-4ce2-b4ec-32aadfd1a089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:09.239390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8a0f8-4738-4ce2-b4ec-32aadfd1a089 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.239677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8a0f8-4738-4ce2-b4ec-32aadfd1a089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.240788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8a0f8-4738-4ce2-b4ec-32aadfd1a089 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.240788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8a0f8-4738-4ce2-b4ec-32aadfd1a089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:09.243515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8a0f8-4738-4ce2-b4ec-32aadfd1a089 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:09.271262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1074/2086] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:56:09 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:09.275053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be5997d6d7854f10931ab97c63fd76f1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.275194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8a0f8-4738-4ce2-b4ec-32aadfd1a089 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:09.275788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be5997d6d7854f10931ab97c63fd76f1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.276432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be5997d6d7854f10931ab97c63fd76f1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.288861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be5997d6d7854f10931ab97c63fd76f1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.296352 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be5997d6d7854f10931ab97c63fd76f1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.312830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1013/2087] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:09 2022] DELETE /identity/v3/users/be5997d6d7854f10931ab97c63fd76f1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:56:09.341799 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:09.346008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8a0f8-4738-4ce2-b4ec-32aadfd1a089 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:09.410287 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487d1a6-091e-4e2d-97d3-eb6e762c7ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:09.411129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d1a6-091e-4e2d-97d3-eb6e762c7ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.411467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d1a6-091e-4e2d-97d3-eb6e762c7ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.411759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d1a6-091e-4e2d-97d3-eb6e762c7ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.412476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487d1a6-091e-4e2d-97d3-eb6e762c7ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:09.415056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487d1a6-091e-4e2d-97d3-eb6e762c7ca6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:09.439028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1075/2088] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:56:09 2022] POST /identity/v3/users => generated 283 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:56:09.443145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:09.444692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:09.458494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5c8f46-2ef6-4409-933e-937e01c7ef37 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:09.471951 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d63e3-c01e-43f9-b59a-644116a48b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:09.472713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d63e3-c01e-43f9-b59a-644116a48b33 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.473000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d63e3-c01e-43f9-b59a-644116a48b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.473283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d63e3-c01e-43f9-b59a-644116a48b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.474030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d63e3-c01e-43f9-b59a-644116a48b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:09.476262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d63e3-c01e-43f9-b59a-644116a48b33 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:09.488732 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1014/2089] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:09.492246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487d1a6-091e-4e2d-97d3-eb6e762c7ca6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:09.504986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487d1a6-091e-4e2d-97d3-eb6e762c7ca6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:09.511621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dc415-911b-449b-8029-dfd9a107df2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:09.512299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dc415-911b-449b-8029-dfd9a107df2c admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.512592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dc415-911b-449b-8029-dfd9a107df2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.512898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dc415-911b-449b-8029-dfd9a107df2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.513638 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dc415-911b-449b-8029-dfd9a107df2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:09.515246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dc415-911b-449b-8029-dfd9a107df2c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:09.528842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1076/2090] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:09.532872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d63e3-c01e-43f9-b59a-644116a48b33 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:09.550314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d63e3-c01e-43f9-b59a-644116a48b33 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:09.556122 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6173e-c62a-4768-ac69-3dbd3a149088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:09.557189 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6173e-c62a-4768-ac69-3dbd3a149088 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.557550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6173e-c62a-4768-ac69-3dbd3a149088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.557906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6173e-c62a-4768-ac69-3dbd3a149088 admin admin] PATH_INFO: `/v3/projects/67adea7232d041a19014f0a75c361a18/users/d617fda3ed0a4a11a90bcae8b714dfab/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.582244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6173e-c62a-4768-ac69-3dbd3a149088 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67adea7232d041a19014f0a75c361a18, user_id=d617fda3ed0a4a11a90bcae8b714dfab, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:09.586679 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6173e-c62a-4768-ac69-3dbd3a149088 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:09.607730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1015/2091] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:09 2022] PUT /identity/v3/projects/67adea7232d041a19014f0a75c361a18/users/d617fda3ed0a4a11a90bcae8b714dfab/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:56:09.620439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffb71d-ef01-48b9-b5e4-938f654adb8b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.621123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffb71d-ef01-48b9-b5e4-938f654adb8b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.621610 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffb71d-ef01-48b9-b5e4-938f654adb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.669100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-66ffb71d-ef01-48b9-b5e4-938f654adb8b None None] MFA Rules not processed for user `d617fda3ed0a4a11a90bcae8b714dfab`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:09.671228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-66ffb71d-ef01-48b9-b5e4-938f654adb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:09.760068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6173e-c62a-4768-ac69-3dbd3a149088 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:09.763155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-13e6173e-c62a-4768-ac69-3dbd3a149088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:09.813948 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1077/2092] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:09.825659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffb71d-ef01-48b9-b5e4-938f654adb8b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:09.827484 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-66ffb71d-ef01-48b9-b5e4-938f654adb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:09.859311 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6173e-c62a-4768-ac69-3dbd3a149088 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:09.864486 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd703fc5-4b93-4866-90c5-63b05706d880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:09.865555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd703fc5-4b93-4866-90c5-63b05706d880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.865952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd703fc5-4b93-4866-90c5-63b05706d880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.866297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd703fc5-4b93-4866-90c5-63b05706d880 admin admin] PATH_INFO: `/v3/projects/0860a96ad3844945b4a9960a7c5c4c87` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.868177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd703fc5-4b93-4866-90c5-63b05706d880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0860a96ad3844945b4a9960a7c5c4c87)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:09.870331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd703fc5-4b93-4866-90c5-63b05706d880 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:09.918928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffb71d-ef01-48b9-b5e4-938f654adb8b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:09.926154 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc1e68-b17e-4af2-810d-16855cde59e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:09.926861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc1e68-b17e-4af2-810d-16855cde59e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:09.927242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc1e68-b17e-4af2-810d-16855cde59e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:09.927580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc1e68-b17e-4af2-810d-16855cde59e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:09.939157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc1e68-b17e-4af2-810d-16855cde59e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:09.940843 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cc1e68-b17e-4af2-810d-16855cde59e9 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:09.942090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:09.955338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1078/2093] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:56:09.955871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bd703fc5-4b93-4866-90c5-63b05706d880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0860a96ad3844945b4a9960a7c5c4c87', 'request_id': 'req-bd703fc5-4b93-4866-90c5-63b05706d880'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.971762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bd703fc5-4b93-4866-90c5-63b05706d880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0860a96ad3844945b4a9960a7c5c4c87', 'request_id': 'req-bd703fc5-4b93-4866-90c5-63b05706d880'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.980512 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bd703fc5-4b93-4866-90c5-63b05706d880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0860a96ad3844945b4a9960a7c5c4c87', 'request_id': 'req-bd703fc5-4b93-4866-90c5-63b05706d880'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.998098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bd703fc5-4b93-4866-90c5-63b05706d880 admin admin] The token cache is being invalidate because project 0860a96ad3844945b4a9960a7c5c4c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:09.998398 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bd703fc5-4b93-4866-90c5-63b05706d880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:09.999108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bd703fc5-4b93-4866-90c5-63b05706d880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:10.001056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1016/2094] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:09 2022] DELETE /identity/v3/projects/0860a96ad3844945b4a9960a7c5c4c87 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:56:10.011194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b9848-e640-4291-ab11-6c0a8e1244bf None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:10.011512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b9848-e640-4291-ab11-6c0a8e1244bf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:10.011789 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b9848-e640-4291-ab11-6c0a8e1244bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:10.053676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f75b9848-e640-4291-ab11-6c0a8e1244bf None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:10.055052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f75b9848-e640-4291-ab11-6c0a8e1244bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:10.073954 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:10.159395 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1079/2095] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:10.167227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66327f-9ffa-486b-b52a-2ca9749adfd5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:10.167521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66327f-9ffa-486b-b52a-2ca9749adfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:10.167806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66327f-9ffa-486b-b52a-2ca9749adfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:10.190586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7b66327f-9ffa-486b-b52a-2ca9749adfd5 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:10.191845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7b66327f-9ffa-486b-b52a-2ca9749adfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:10.202079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1017/2096] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:10.205577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b9848-e640-4291-ab11-6c0a8e1244bf None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:10.215744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b9848-e640-4291-ab11-6c0a8e1244bf None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:10.220874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff224fbf-9f3a-4fba-93a8-b9eed6002a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:10.221616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff224fbf-9f3a-4fba-93a8-b9eed6002a15 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:10.221961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff224fbf-9f3a-4fba-93a8-b9eed6002a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:10.222302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff224fbf-9f3a-4fba-93a8-b9eed6002a15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:10.223112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff224fbf-9f3a-4fba-93a8-b9eed6002a15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:56:10.223492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff224fbf-9f3a-4fba-93a8-b9eed6002a15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:10.225267 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff224fbf-9f3a-4fba-93a8-b9eed6002a15 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:10.244720 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1080/2097] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:56:10.246959 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66327f-9ffa-486b-b52a-2ca9749adfd5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:10.257247 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66327f-9ffa-486b-b52a-2ca9749adfd5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:10.262274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d599d8a-7735-4a69-9b83-78d97752745d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:10.262966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d599d8a-7735-4a69-9b83-78d97752745d admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:10.263274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d599d8a-7735-4a69-9b83-78d97752745d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:10.263559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d599d8a-7735-4a69-9b83-78d97752745d admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:10.264294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d599d8a-7735-4a69-9b83-78d97752745d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:10.266227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d599d8a-7735-4a69-9b83-78d97752745d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:10.309469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1018/2098] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:56:10 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:56:10.313293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff224fbf-9f3a-4fba-93a8-b9eed6002a15 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:10.325160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff224fbf-9f3a-4fba-93a8-b9eed6002a15 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:10.394334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e36fe0-68ac-43c9-84eb-32a46bf37412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:10.395001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e36fe0-68ac-43c9-84eb-32a46bf37412 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:10.395289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e36fe0-68ac-43c9-84eb-32a46bf37412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:10.395570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e36fe0-68ac-43c9-84eb-32a46bf37412 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:10.396260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e36fe0-68ac-43c9-84eb-32a46bf37412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:10.398298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e36fe0-68ac-43c9-84eb-32a46bf37412 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:10.418612 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1081/2099] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:56:10 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:56:10.423034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d599d8a-7735-4a69-9b83-78d97752745d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:10.433953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d599d8a-7735-4a69-9b83-78d97752745d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:10.438610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b5be6-f4cb-4310-9ac8-d34a5e043856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:10.439270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b5be6-f4cb-4310-9ac8-d34a5e043856 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:10.440166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b5be6-f4cb-4310-9ac8-d34a5e043856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:10.440457 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b5be6-f4cb-4310-9ac8-d34a5e043856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:10.441234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b5be6-f4cb-4310-9ac8-d34a5e043856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:10.442835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b5be6-f4cb-4310-9ac8-d34a5e043856 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:10.454559 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1019/2100] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:10.458051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e36fe0-68ac-43c9-84eb-32a46bf37412 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:10.467959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e36fe0-68ac-43c9-84eb-32a46bf37412 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:10.472606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8436ed-c605-43fe-b5ad-9c601bccbe52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:10.473288 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8436ed-c605-43fe-b5ad-9c601bccbe52 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:10.473573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8436ed-c605-43fe-b5ad-9c601bccbe52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:10.473849 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8436ed-c605-43fe-b5ad-9c601bccbe52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:10.474579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8436ed-c605-43fe-b5ad-9c601bccbe52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:10.476177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8436ed-c605-43fe-b5ad-9c601bccbe52 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:10.486045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1082/2101] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:10.490309 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b5be6-f4cb-4310-9ac8-d34a5e043856 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:10.501053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b5be6-f4cb-4310-9ac8-d34a5e043856 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:10.505721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023462e-d056-4d15-8383-82f8829ff0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:10.507234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023462e-d056-4d15-8383-82f8829ff0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:10.507523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023462e-d056-4d15-8383-82f8829ff0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:10.507802 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023462e-d056-4d15-8383-82f8829ff0c4 admin admin] PATH_INFO: `/v3/projects/ad3973cbb6b5400fa0693413aa55eee3/users/48d2334d3a1741849025c50c8c456e4e/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:10.522277 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023462e-d056-4d15-8383-82f8829ff0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad3973cbb6b5400fa0693413aa55eee3, user_id=48d2334d3a1741849025c50c8c456e4e, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:10.524953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023462e-d056-4d15-8383-82f8829ff0c4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:10.538748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1020/2102] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:10 2022] PUT /identity/v3/projects/ad3973cbb6b5400fa0693413aa55eee3/users/48d2334d3a1741849025c50c8c456e4e/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:10.554306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8df4b4-f152-4d8c-b924-93e5e4fa7627 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:10.554764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8df4b4-f152-4d8c-b924-93e5e4fa7627 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:10.555649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8df4b4-f152-4d8c-b924-93e5e4fa7627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:10.581989 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bd8df4b4-f152-4d8c-b924-93e5e4fa7627 None None] MFA Rules not processed for user `48d2334d3a1741849025c50c8c456e4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:10.583215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bd8df4b4-f152-4d8c-b924-93e5e4fa7627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:10.650841 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1083/2103] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:10.656082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023462e-d056-4d15-8383-82f8829ff0c4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:10.750073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023462e-d056-4d15-8383-82f8829ff0c4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:10.758051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf3090-4420-48f1-ac94-603749ddb14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:10.759095 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf3090-4420-48f1-ac94-603749ddb14f admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:10.759565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf3090-4420-48f1-ac94-603749ddb14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:10.760033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf3090-4420-48f1-ac94-603749ddb14f admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:10.761277 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf3090-4420-48f1-ac94-603749ddb14f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:10.764306 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf3090-4420-48f1-ac94-603749ddb14f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:10.820839 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1021/2104] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:56:10 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:10.824417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8df4b4-f152-4d8c-b924-93e5e4fa7627 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:10.836737 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8df4b4-f152-4d8c-b924-93e5e4fa7627 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:10.841763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:10.916188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3034f75-d527-42ca-bf17-156f2ffb57b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:10.916921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3034f75-d527-42ca-bf17-156f2ffb57b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:10.917213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3034f75-d527-42ca-bf17-156f2ffb57b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:10.917477 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3034f75-d527-42ca-bf17-156f2ffb57b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:10.918190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3034f75-d527-42ca-bf17-156f2ffb57b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:10.920464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3034f75-d527-42ca-bf17-156f2ffb57b7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:10.946230 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1084/2105] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:56:10 2022] POST /identity/v3/users => generated 295 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:10.949328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf3090-4420-48f1-ac94-603749ddb14f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:10.965196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf3090-4420-48f1-ac94-603749ddb14f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:10.970159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1c4c9-802c-451f-ae21-d0af56e594d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:10.970837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1c4c9-802c-451f-ae21-d0af56e594d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:10.971131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1c4c9-802c-451f-ae21-d0af56e594d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:10.971433 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1c4c9-802c-451f-ae21-d0af56e594d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:10.972150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1c4c9-802c-451f-ae21-d0af56e594d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:10.973811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1c4c9-802c-451f-ae21-d0af56e594d0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:10.985565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1022/2106] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:10.989980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3034f75-d527-42ca-bf17-156f2ffb57b7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:11.001238 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3034f75-d527-42ca-bf17-156f2ffb57b7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:11.006228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8f436-0138-46f2-912e-dfa1706a0990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:11.007139 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8f436-0138-46f2-912e-dfa1706a0990 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:11.007532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8f436-0138-46f2-912e-dfa1706a0990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:11.007936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce8f436-0138-46f2-912e-dfa1706a0990 admin admin] PATH_INFO: `/v3/projects/a8c6d47841814a589732782f95abcba0/users/88538cd987614a8385e3856f31221722/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:11.025229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8f436-0138-46f2-912e-dfa1706a0990 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c6d47841814a589732782f95abcba0, user_id=88538cd987614a8385e3856f31221722, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:11.027114 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce8f436-0138-46f2-912e-dfa1706a0990 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:11.049271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1085/2107] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:10 2022] PUT /identity/v3/projects/a8c6d47841814a589732782f95abcba0/users/88538cd987614a8385e3856f31221722/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:11.053528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1c4c9-802c-451f-ae21-d0af56e594d0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:11.066504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1c4c9-802c-451f-ae21-d0af56e594d0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:11.108919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:11.159967 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c9106-a894-488b-ac91-e628b15e1507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:11.160969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c9106-a894-488b-ac91-e628b15e1507 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:11.161445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c9106-a894-488b-ac91-e628b15e1507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:11.162014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c9106-a894-488b-ac91-e628b15e1507 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:11.163540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c9106-a894-488b-ac91-e628b15e1507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:11.166964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c9106-a894-488b-ac91-e628b15e1507 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:11.189595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1023/2108] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:11.192549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8f436-0138-46f2-912e-dfa1706a0990 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:11.211943 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce8f436-0138-46f2-912e-dfa1706a0990 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:11.222006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148033c-c52b-4b73-b4ef-aec56eb54fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:11.223584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148033c-c52b-4b73-b4ef-aec56eb54fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:11.224117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148033c-c52b-4b73-b4ef-aec56eb54fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:11.224614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148033c-c52b-4b73-b4ef-aec56eb54fc6 admin admin] PATH_INFO: `/v3/projects/a8c6d47841814a589732782f95abcba0/users/88538cd987614a8385e3856f31221722/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:11.229162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148033c-c52b-4b73-b4ef-aec56eb54fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c6d47841814a589732782f95abcba0, user_id=88538cd987614a8385e3856f31221722, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:11.232104 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148033c-c52b-4b73-b4ef-aec56eb54fc6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:11.244432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1086/2109] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:11 2022] PUT /identity/v3/projects/a8c6d47841814a589732782f95abcba0/users/88538cd987614a8385e3856f31221722/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:11.255322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff635f-adf7-47e3-92f8-e21520ad0ba3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:11.255865 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff635f-adf7-47e3-92f8-e21520ad0ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:11.255865 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff635f-adf7-47e3-92f8-e21520ad0ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:11.285883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8aff635f-adf7-47e3-92f8-e21520ad0ba3 None None] MFA Rules not processed for user `88538cd987614a8385e3856f31221722`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:11.287249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8aff635f-adf7-47e3-92f8-e21520ad0ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:11.362305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1024/2110] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:11.376024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148033c-c52b-4b73-b4ef-aec56eb54fc6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:11.378114 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1148033c-c52b-4b73-b4ef-aec56eb54fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:11.438989 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148033c-c52b-4b73-b4ef-aec56eb54fc6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:11.442679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600327f-9157-48df-976f-30e9055d1403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:11.443338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600327f-9157-48df-976f-30e9055d1403 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:11.443630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600327f-9157-48df-976f-30e9055d1403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:11.443909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600327f-9157-48df-976f-30e9055d1403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:11.505736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600327f-9157-48df-976f-30e9055d1403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:11.507806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600327f-9157-48df-976f-30e9055d1403 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:11.519036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1087/2111] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:56:17.256059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff635f-adf7-47e3-92f8-e21520ad0ba3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:17.266681 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff635f-adf7-47e3-92f8-e21520ad0ba3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:17.270519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501f0de-4993-4cd6-a2a7-eea00aa98118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:17.271153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501f0de-4993-4cd6-a2a7-eea00aa98118 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:17.271436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501f0de-4993-4cd6-a2a7-eea00aa98118 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:17.271713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501f0de-4993-4cd6-a2a7-eea00aa98118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:17.281187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501f0de-4993-4cd6-a2a7-eea00aa98118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:17.282608 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501f0de-4993-4cd6-a2a7-eea00aa98118 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:17.292418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1025/2112] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:56:21.514825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:27.287650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=53 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:39.174310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d4d65-4635-41f6-9d0e-8aa3cf990812 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:39.174960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d4d65-4635-41f6-9d0e-8aa3cf990812 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:39.174960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d4d65-4635-41f6-9d0e-8aa3cf990812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:39.193599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9c0d4d65-4635-41f6-9d0e-8aa3cf990812 None None] MFA Rules not processed for user `23a9ad25030045b9b5dad89eadca1b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:39.194842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9c0d4d65-4635-41f6-9d0e-8aa3cf990812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:39.252470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1088/2113] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:49.245182 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=25 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:50.148826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501f0de-4993-4cd6-a2a7-eea00aa98118 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:50.215021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501f0de-4993-4cd6-a2a7-eea00aa98118 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:50.219587 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:50.220451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:50.220763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:50.221043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] PATH_INFO: `/v3/users/48d2334d3a1741849025c50c8c456e4e` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:50.222248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48d2334d3a1741849025c50c8c456e4e)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:50.224323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:50.276413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48d2334d3a1741849025c50c8c456e4e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:50.277255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48d2334d3a1741849025c50c8c456e4e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:50.277972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48d2334d3a1741849025c50c8c456e4e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:50.293640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48d2334d3a1741849025c50c8c456e4e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:50.301749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48d2334d3a1741849025c50c8c456e4e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:50.326778 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1026/2114] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:50 2022] DELETE /identity/v3/users/48d2334d3a1741849025c50c8c456e4e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:50.331255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6c686-eb86-467c-8f7e-79ee7a831cf9 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:50.331955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6c686-eb86-467c-8f7e-79ee7a831cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:50.331955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6c686-eb86-467c-8f7e-79ee7a831cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:50.352630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-17a6c686-eb86-467c-8f7e-79ee7a831cf9 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:50.353949 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-17a6c686-eb86-467c-8f7e-79ee7a831cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:50.430565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1089/2115] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:56:50.440967 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:50.442656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:50.509351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e312d9f-e93c-45e5-8a6d-ffd45ff35551 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:50.517653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71bcfc-747e-4b64-8680-1da4a462dea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:50.518384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71bcfc-747e-4b64-8680-1da4a462dea0 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:50.518686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71bcfc-747e-4b64-8680-1da4a462dea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:50.518969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71bcfc-747e-4b64-8680-1da4a462dea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:50.529669 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71bcfc-747e-4b64-8680-1da4a462dea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:50.531225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71bcfc-747e-4b64-8680-1da4a462dea0 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:50.542733 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1027/2116] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:56:50.819006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6c686-eb86-467c-8f7e-79ee7a831cf9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:50.820570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-17a6c686-eb86-467c-8f7e-79ee7a831cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:50.834309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6c686-eb86-467c-8f7e-79ee7a831cf9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:50.840476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:50.841547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:50.841928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:50.842230 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] PATH_INFO: `/v3/projects/ad3973cbb6b5400fa0693413aa55eee3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:50.845546 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad3973cbb6b5400fa0693413aa55eee3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:50.847772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:50.901958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad3973cbb6b5400fa0693413aa55eee3', 'request_id': 'req-dce9daaa-98ab-48f3-992a-7e71953002da'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:50.910410 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad3973cbb6b5400fa0693413aa55eee3', 'request_id': 'req-dce9daaa-98ab-48f3-992a-7e71953002da'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:50.918031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad3973cbb6b5400fa0693413aa55eee3', 'request_id': 'req-dce9daaa-98ab-48f3-992a-7e71953002da'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:50.930074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] The token cache is being invalidate because project ad3973cbb6b5400fa0693413aa55eee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:50.930433 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:50.931112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:50.932755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1090/2117] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:50 2022] DELETE /identity/v3/projects/ad3973cbb6b5400fa0693413aa55eee3 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:50.936642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71bcfc-747e-4b64-8680-1da4a462dea0 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:50.946796 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71bcfc-747e-4b64-8680-1da4a462dea0 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:50.967291 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:51.008204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3583d-5715-471c-86d3-81c18e5c91f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:51.009119 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3583d-5715-471c-86d3-81c18e5c91f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:51.009409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3583d-5715-471c-86d3-81c18e5c91f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:51.009692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3583d-5715-471c-86d3-81c18e5c91f4 admin admin] PATH_INFO: `/v3/users/88538cd987614a8385e3856f31221722` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:51.011416 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3583d-5715-471c-86d3-81c18e5c91f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88538cd987614a8385e3856f31221722)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:51.013399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3583d-5715-471c-86d3-81c18e5c91f4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:51.074981 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-49b3583d-5715-471c-86d3-81c18e5c91f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88538cd987614a8385e3856f31221722'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:51.075665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-49b3583d-5715-471c-86d3-81c18e5c91f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88538cd987614a8385e3856f31221722'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:51.076305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-49b3583d-5715-471c-86d3-81c18e5c91f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88538cd987614a8385e3856f31221722'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:51.090035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-49b3583d-5715-471c-86d3-81c18e5c91f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88538cd987614a8385e3856f31221722'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:51.097749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-49b3583d-5715-471c-86d3-81c18e5c91f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88538cd987614a8385e3856f31221722'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:51.111584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1028/2118] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:50 2022] DELETE /identity/v3/users/88538cd987614a8385e3856f31221722 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:51.390339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:51.391869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:51.406215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9daaa-98ab-48f3-992a-7e71953002da admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:51.474030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e46f97-cc36-4454-8718-aa2f021201c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:51.474901 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e46f97-cc36-4454-8718-aa2f021201c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:51.475190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e46f97-cc36-4454-8718-aa2f021201c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:51.475470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e46f97-cc36-4454-8718-aa2f021201c8 admin admin] PATH_INFO: `/v3/projects/a8c6d47841814a589732782f95abcba0` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:51.476692 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e46f97-cc36-4454-8718-aa2f021201c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c6d47841814a589732782f95abcba0)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:51.478617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e46f97-cc36-4454-8718-aa2f021201c8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:51.509931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:51.528319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-08e46f97-cc36-4454-8718-aa2f021201c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c6d47841814a589732782f95abcba0', 'request_id': 'req-08e46f97-cc36-4454-8718-aa2f021201c8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:51.536401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-08e46f97-cc36-4454-8718-aa2f021201c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c6d47841814a589732782f95abcba0', 'request_id': 'req-08e46f97-cc36-4454-8718-aa2f021201c8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:51.542574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-08e46f97-cc36-4454-8718-aa2f021201c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c6d47841814a589732782f95abcba0', 'request_id': 'req-08e46f97-cc36-4454-8718-aa2f021201c8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:51.561679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-08e46f97-cc36-4454-8718-aa2f021201c8 admin admin] The token cache is being invalidate because project a8c6d47841814a589732782f95abcba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:51.561981 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-08e46f97-cc36-4454-8718-aa2f021201c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:51.562691 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-08e46f97-cc36-4454-8718-aa2f021201c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:51.564276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1091/2119] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:51 2022] DELETE /identity/v3/projects/a8c6d47841814a589732782f95abcba0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:51.569207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98447a-d867-4f83-b5ce-98ddeebfddfd None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:51.569697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98447a-d867-4f83-b5ce-98ddeebfddfd None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:51.569795 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98447a-d867-4f83-b5ce-98ddeebfddfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:51.587621 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-af98447a-d867-4f83-b5ce-98ddeebfddfd None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:51.590266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-af98447a-d867-4f83-b5ce-98ddeebfddfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:51.662567 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1029/2120] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:51.671872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf23ba-4c2c-4f0b-8ba9-d6361e0acd9a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:51.672010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf23ba-4c2c-4f0b-8ba9-d6361e0acd9a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:51.672276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf23ba-4c2c-4f0b-8ba9-d6361e0acd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:51.691742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-73cf23ba-4c2c-4f0b-8ba9-d6361e0acd9a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:51.693059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-73cf23ba-4c2c-4f0b-8ba9-d6361e0acd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:51.702423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1092/2121] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:51.706850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98447a-d867-4f83-b5ce-98ddeebfddfd None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:51.717359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98447a-d867-4f83-b5ce-98ddeebfddfd None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:51.722561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411e1d7-89c9-4d64-a82f-72a18afa5a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:51.723240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411e1d7-89c9-4d64-a82f-72a18afa5a8d admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:51.723532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411e1d7-89c9-4d64-a82f-72a18afa5a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:51.723931 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5411e1d7-89c9-4d64-a82f-72a18afa5a8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:51.724759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411e1d7-89c9-4d64-a82f-72a18afa5a8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:56:51.725035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411e1d7-89c9-4d64-a82f-72a18afa5a8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:51.726857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5411e1d7-89c9-4d64-a82f-72a18afa5a8d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:51.740217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1030/2122] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:56:51.744450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf23ba-4c2c-4f0b-8ba9-d6361e0acd9a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:51.754892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf23ba-4c2c-4f0b-8ba9-d6361e0acd9a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:51.759967 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80b97b-43f2-4aa8-9b5d-4e3135b5b555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:51.761252 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80b97b-43f2-4aa8-9b5d-4e3135b5b555 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:51.761538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80b97b-43f2-4aa8-9b5d-4e3135b5b555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:51.761819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f80b97b-43f2-4aa8-9b5d-4e3135b5b555 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:51.762528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80b97b-43f2-4aa8-9b5d-4e3135b5b555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:51.764481 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f80b97b-43f2-4aa8-9b5d-4e3135b5b555 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:51.792099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1093/2123] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:56:51 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:51.797099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411e1d7-89c9-4d64-a82f-72a18afa5a8d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:51.807200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5411e1d7-89c9-4d64-a82f-72a18afa5a8d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:51.834195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:51.867808 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21c4f4-420f-4fa2-b492-98c9f12f4836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:51.868487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21c4f4-420f-4fa2-b492-98c9f12f4836 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:51.868800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21c4f4-420f-4fa2-b492-98c9f12f4836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:51.869080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21c4f4-420f-4fa2-b492-98c9f12f4836 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:51.869827 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21c4f4-420f-4fa2-b492-98c9f12f4836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:51.871758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21c4f4-420f-4fa2-b492-98c9f12f4836 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:51.892540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1031/2124] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:56:51 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:51.896846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80b97b-43f2-4aa8-9b5d-4e3135b5b555 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:51.908190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f80b97b-43f2-4aa8-9b5d-4e3135b5b555 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:51.912885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354a328-e5ea-4c49-a4cd-02cd128953e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:51.913547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354a328-e5ea-4c49-a4cd-02cd128953e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:51.913835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354a328-e5ea-4c49-a4cd-02cd128953e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:51.914113 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6354a328-e5ea-4c49-a4cd-02cd128953e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:51.914846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354a328-e5ea-4c49-a4cd-02cd128953e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:51.916442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6354a328-e5ea-4c49-a4cd-02cd128953e1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:51.928197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1094/2125] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:51.933063 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21c4f4-420f-4fa2-b492-98c9f12f4836 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:51.943743 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21c4f4-420f-4fa2-b492-98c9f12f4836 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:51.949172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e1994-259c-4112-9fd1-5f73c54caf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:51.949819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e1994-259c-4112-9fd1-5f73c54caf2e admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:51.950118 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e1994-259c-4112-9fd1-5f73c54caf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:51.950397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e1994-259c-4112-9fd1-5f73c54caf2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:51.951124 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e1994-259c-4112-9fd1-5f73c54caf2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:51.952759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e1994-259c-4112-9fd1-5f73c54caf2e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:51.963374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1032/2126] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:51.968104 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354a328-e5ea-4c49-a4cd-02cd128953e1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:51.979270 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6354a328-e5ea-4c49-a4cd-02cd128953e1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:51.984424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4b585-60cf-4d31-9c5b-971d6db0a42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:51.985470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4b585-60cf-4d31-9c5b-971d6db0a42a admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:51.985630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4b585-60cf-4d31-9c5b-971d6db0a42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:51.985912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4b585-60cf-4d31-9c5b-971d6db0a42a admin admin] PATH_INFO: `/v3/projects/2bf39ffbd9e247b3a9743b71fd8a5d1d/users/506c6f39c97b4c8b8ffc8331e2bb174a/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:52.001092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4b585-60cf-4d31-9c5b-971d6db0a42a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf39ffbd9e247b3a9743b71fd8a5d1d, user_id=506c6f39c97b4c8b8ffc8331e2bb174a, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:52.003962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4b585-60cf-4d31-9c5b-971d6db0a42a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:52.020520 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1095/2127] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:51 2022] PUT /identity/v3/projects/2bf39ffbd9e247b3a9743b71fd8a5d1d/users/506c6f39c97b4c8b8ffc8331e2bb174a/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:52.031426 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f3267-4aa9-4b7b-b4f6-8484babd325d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:52.031668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f3267-4aa9-4b7b-b4f6-8484babd325d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:52.031883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f3267-4aa9-4b7b-b4f6-8484babd325d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:52.060646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-319f3267-4aa9-4b7b-b4f6-8484babd325d None None] MFA Rules not processed for user `506c6f39c97b4c8b8ffc8331e2bb174a`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:52.061975 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-319f3267-4aa9-4b7b-b4f6-8484babd325d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:52.141547 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1033/2128] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:52.154159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4b585-60cf-4d31-9c5b-971d6db0a42a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:52.229669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4b585-60cf-4d31-9c5b-971d6db0a42a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:52.234950 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c0e05-d949-4c2b-a078-cc1b765c7e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:52.235628 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c0e05-d949-4c2b-a078-cc1b765c7e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:52.235922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c0e05-d949-4c2b-a078-cc1b765c7e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:52.236203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c0e05-d949-4c2b-a078-cc1b765c7e18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:52.236952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c0e05-d949-4c2b-a078-cc1b765c7e18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:52.238896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c0e05-d949-4c2b-a078-cc1b765c7e18 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:52.266874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1096/2129] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:56:52 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:52.271260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f3267-4aa9-4b7b-b4f6-8484babd325d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:52.282721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f3267-4aa9-4b7b-b4f6-8484babd325d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:52.340944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30416824-b842-4f37-8868-38bd420bf18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:52.341623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30416824-b842-4f37-8868-38bd420bf18e admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:52.341915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30416824-b842-4f37-8868-38bd420bf18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:52.342200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30416824-b842-4f37-8868-38bd420bf18e admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:52.342899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30416824-b842-4f37-8868-38bd420bf18e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:52.344892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30416824-b842-4f37-8868-38bd420bf18e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:52.366443 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1034/2130] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:56:52 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:52.370981 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c0e05-d949-4c2b-a078-cc1b765c7e18 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:52.381598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c0e05-d949-4c2b-a078-cc1b765c7e18 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:52.386616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed5570-3a4c-4285-b7df-984cc53f8e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:52.387286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed5570-3a4c-4285-b7df-984cc53f8e05 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:52.387607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed5570-3a4c-4285-b7df-984cc53f8e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:52.387907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed5570-3a4c-4285-b7df-984cc53f8e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:52.388623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed5570-3a4c-4285-b7df-984cc53f8e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:52.390274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed5570-3a4c-4285-b7df-984cc53f8e05 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:52.403888 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1097/2131] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:52.407984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30416824-b842-4f37-8868-38bd420bf18e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:52.418834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30416824-b842-4f37-8868-38bd420bf18e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:52.424638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d478bc-fe22-4188-8ba6-3d2ba49ddcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:52.425548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d478bc-fe22-4188-8ba6-3d2ba49ddcb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:52.425832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d478bc-fe22-4188-8ba6-3d2ba49ddcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:52.426112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d478bc-fe22-4188-8ba6-3d2ba49ddcb6 admin admin] PATH_INFO: `/v3/projects/a71c5588f64b4766bf623bb55c484d93/users/b226446d7cbf404abc437e09d101ec0b/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:52.441523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d478bc-fe22-4188-8ba6-3d2ba49ddcb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71c5588f64b4766bf623bb55c484d93, user_id=b226446d7cbf404abc437e09d101ec0b, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:52.444187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d478bc-fe22-4188-8ba6-3d2ba49ddcb6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:52.458752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1035/2132] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:52 2022] PUT /identity/v3/projects/a71c5588f64b4766bf623bb55c484d93/users/b226446d7cbf404abc437e09d101ec0b/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:56:52.462770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed5570-3a4c-4285-b7df-984cc53f8e05 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:52.476141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed5570-3a4c-4285-b7df-984cc53f8e05 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:52.498741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:52.538303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb4696-017a-42ec-ad63-86e7c0de52b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:52.538974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb4696-017a-42ec-ad63-86e7c0de52b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:52.539265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb4696-017a-42ec-ad63-86e7c0de52b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:52.539550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb4696-017a-42ec-ad63-86e7c0de52b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:52.540285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb4696-017a-42ec-ad63-86e7c0de52b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:52.541930 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb4696-017a-42ec-ad63-86e7c0de52b1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:52.551941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1098/2133] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:52.556051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d478bc-fe22-4188-8ba6-3d2ba49ddcb6 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:52.561500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:52.567994 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d478bc-fe22-4188-8ba6-3d2ba49ddcb6 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:52.574125 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633b3ca-3479-4364-8b07-4d1ba61d0030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:52.574998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633b3ca-3479-4364-8b07-4d1ba61d0030 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:52.575282 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633b3ca-3479-4364-8b07-4d1ba61d0030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:52.575557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633b3ca-3479-4364-8b07-4d1ba61d0030 admin admin] PATH_INFO: `/v3/projects/a71c5588f64b4766bf623bb55c484d93/users/b226446d7cbf404abc437e09d101ec0b/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:52.577646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633b3ca-3479-4364-8b07-4d1ba61d0030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a71c5588f64b4766bf623bb55c484d93, user_id=b226446d7cbf404abc437e09d101ec0b, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:52.580323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633b3ca-3479-4364-8b07-4d1ba61d0030 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:52.591289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1036/2134] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:52 2022] PUT /identity/v3/projects/a71c5588f64b4766bf623bb55c484d93/users/b226446d7cbf404abc437e09d101ec0b/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:52.601289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e628c6-1cf0-46b7-9821-c5a4d4c45fc0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:52.601587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e628c6-1cf0-46b7-9821-c5a4d4c45fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:52.601870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e628c6-1cf0-46b7-9821-c5a4d4c45fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:52.629327 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e5e628c6-1cf0-46b7-9821-c5a4d4c45fc0 None None] MFA Rules not processed for user `b226446d7cbf404abc437e09d101ec0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:52.630549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e5e628c6-1cf0-46b7-9821-c5a4d4c45fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:52.704001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1099/2135] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:52.715584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633b3ca-3479-4364-8b07-4d1ba61d0030 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:52.717822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3633b3ca-3479-4364-8b07-4d1ba61d0030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:52.788179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633b3ca-3479-4364-8b07-4d1ba61d0030 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:52.792485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1640a6d9-9d1a-4e7a-ac9a-b09c4f763bd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:52.793211 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640a6d9-9d1a-4e7a-ac9a-b09c4f763bd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:52.793495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640a6d9-9d1a-4e7a-ac9a-b09c4f763bd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:52.793770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1640a6d9-9d1a-4e7a-ac9a-b09c4f763bd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:52.804418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640a6d9-9d1a-4e7a-ac9a-b09c4f763bd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:52.805927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1640a6d9-9d1a-4e7a-ac9a-b09c4f763bd6 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:52.816545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1037/2136] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:56:52.844632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e628c6-1cf0-46b7-9821-c5a4d4c45fc0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:52.909900 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e628c6-1cf0-46b7-9821-c5a4d4c45fc0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:52.914584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:52.915460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:52.915747 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:52.916025 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] PATH_INFO: `/v3/users/506c6f39c97b4c8b8ffc8331e2bb174a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:52.917934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=506c6f39c97b4c8b8ffc8331e2bb174a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:52.920126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:52.967182 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '506c6f39c97b4c8b8ffc8331e2bb174a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:52.967765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '506c6f39c97b4c8b8ffc8331e2bb174a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:52.968376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '506c6f39c97b4c8b8ffc8331e2bb174a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:52.969535 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:52.981988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '506c6f39c97b4c8b8ffc8331e2bb174a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:52.989093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '506c6f39c97b4c8b8ffc8331e2bb174a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:53.003774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1100/2137] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:52 2022] DELETE /identity/v3/users/506c6f39c97b4c8b8ffc8331e2bb174a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:53.008363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094b52c-9fca-42d9-8eab-1a2306df1b7f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:53.008794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094b52c-9fca-42d9-8eab-1a2306df1b7f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:53.009173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094b52c-9fca-42d9-8eab-1a2306df1b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:53.027005 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6094b52c-9fca-42d9-8eab-1a2306df1b7f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:53.028204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6094b52c-9fca-42d9-8eab-1a2306df1b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:53.088805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1038/2138] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:56:53.098874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:53.100297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:53.162432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b89228-2d6a-4b7d-bc77-c0fac9e892b4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:53.166856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de923b-8c37-40a2-bac3-7d4d770c1eb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:53.167790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de923b-8c37-40a2-bac3-7d4d770c1eb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:53.168152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de923b-8c37-40a2-bac3-7d4d770c1eb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:53.168574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0de923b-8c37-40a2-bac3-7d4d770c1eb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:53.180905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de923b-8c37-40a2-bac3-7d4d770c1eb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:53.183025 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0de923b-8c37-40a2-bac3-7d4d770c1eb7 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:53.193590 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1101/2139] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:56:54.049825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094b52c-9fca-42d9-8eab-1a2306df1b7f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:54.051309 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6094b52c-9fca-42d9-8eab-1a2306df1b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:54.073958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094b52c-9fca-42d9-8eab-1a2306df1b7f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:54.082698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:54.083551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.083836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.084157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] PATH_INFO: `/v3/projects/2bf39ffbd9e247b3a9743b71fd8a5d1d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.085460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf39ffbd9e247b3a9743b71fd8a5d1d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:54.087410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:54.136486 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf39ffbd9e247b3a9743b71fd8a5d1d', 'request_id': 'req-384116c1-61a9-4170-a03b-7e70f0850866'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.144402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf39ffbd9e247b3a9743b71fd8a5d1d', 'request_id': 'req-384116c1-61a9-4170-a03b-7e70f0850866'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.153252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf39ffbd9e247b3a9743b71fd8a5d1d', 'request_id': 'req-384116c1-61a9-4170-a03b-7e70f0850866'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.165578 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] The token cache is being invalidate because project 2bf39ffbd9e247b3a9743b71fd8a5d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:54.165787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.167214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.169252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1039/2140] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:54 2022] DELETE /identity/v3/projects/2bf39ffbd9e247b3a9743b71fd8a5d1d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:54.173265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de923b-8c37-40a2-bac3-7d4d770c1eb7 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:54.184694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0de923b-8c37-40a2-bac3-7d4d770c1eb7 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:54.250753 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa560ff6-4b67-4bce-a34f-de7ccc55a2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:54.251631 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa560ff6-4b67-4bce-a34f-de7ccc55a2ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.251920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa560ff6-4b67-4bce-a34f-de7ccc55a2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.252203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa560ff6-4b67-4bce-a34f-de7ccc55a2ce admin admin] PATH_INFO: `/v3/users/b226446d7cbf404abc437e09d101ec0b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.253484 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa560ff6-4b67-4bce-a34f-de7ccc55a2ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=b226446d7cbf404abc437e09d101ec0b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:54.255477 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa560ff6-4b67-4bce-a34f-de7ccc55a2ce admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:54.313618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aa560ff6-4b67-4bce-a34f-de7ccc55a2ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b226446d7cbf404abc437e09d101ec0b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.314284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aa560ff6-4b67-4bce-a34f-de7ccc55a2ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b226446d7cbf404abc437e09d101ec0b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.314902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aa560ff6-4b67-4bce-a34f-de7ccc55a2ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b226446d7cbf404abc437e09d101ec0b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.327225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aa560ff6-4b67-4bce-a34f-de7ccc55a2ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b226446d7cbf404abc437e09d101ec0b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.331696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:54.334827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aa560ff6-4b67-4bce-a34f-de7ccc55a2ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b226446d7cbf404abc437e09d101ec0b'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.350422 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1102/2141] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:56:54 2022] DELETE /identity/v3/users/b226446d7cbf404abc437e09d101ec0b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:54.721798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:54.723264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:54.737481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384116c1-61a9-4170-a03b-7e70f0850866 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:54.752332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:54.803605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:54.804473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.804819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.805106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f admin admin] PATH_INFO: `/v3/projects/a71c5588f64b4766bf623bb55c484d93` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.806356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a71c5588f64b4766bf623bb55c484d93)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:54.809265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:54.856902 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a71c5588f64b4766bf623bb55c484d93', 'request_id': 'req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.864594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a71c5588f64b4766bf623bb55c484d93', 'request_id': 'req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.871808 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a71c5588f64b4766bf623bb55c484d93', 'request_id': 'req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.882210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f admin admin] The token cache is being invalidate because project a71c5588f64b4766bf623bb55c484d93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:56:54.882501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.883150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4c9b0624-921c-4e35-8ceb-bdf5fba5129f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:56:54.884715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1040/2142] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:56:54 2022] DELETE /identity/v3/projects/a71c5588f64b4766bf623bb55c484d93 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:54.889715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a1a83-bd76-4580-a49d-40527044b4a0 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.890010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a1a83-bd76-4580-a49d-40527044b4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.890294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a1a83-bd76-4580-a49d-40527044b4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.909379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-df0a1a83-bd76-4580-a49d-40527044b4a0 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:54.910579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-df0a1a83-bd76-4580-a49d-40527044b4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:54.969943 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1103/2143] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:56:54.978684 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa12e2c-910a-437c-9b98-82f355857dec None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:54.979549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa12e2c-910a-437c-9b98-82f355857dec None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:54.979839 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa12e2c-910a-437c-9b98-82f355857dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:54.999008 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-faa12e2c-910a-437c-9b98-82f355857dec None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:55.000279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-faa12e2c-910a-437c-9b98-82f355857dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:55.008456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1041/2144] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:55.013274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a1a83-bd76-4580-a49d-40527044b4a0 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.023142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a1a83-bd76-4580-a49d-40527044b4a0 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.028341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebc1c4-a02d-4368-943e-f3d2906828f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.029044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebc1c4-a02d-4368-943e-f3d2906828f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.029332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebc1c4-a02d-4368-943e-f3d2906828f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.029614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebc1c4-a02d-4368-943e-f3d2906828f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.030355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebc1c4-a02d-4368-943e-f3d2906828f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:56:55.030670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebc1c4-a02d-4368-943e-f3d2906828f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.032280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebc1c4-a02d-4368-943e-f3d2906828f1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.044049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1104/2145] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:56:55.048526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa12e2c-910a-437c-9b98-82f355857dec None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.058606 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa12e2c-910a-437c-9b98-82f355857dec None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.063474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b08522-228c-4fda-a7c7-ae0dda82813d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.064130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b08522-228c-4fda-a7c7-ae0dda82813d admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.064417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b08522-228c-4fda-a7c7-ae0dda82813d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.064752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b08522-228c-4fda-a7c7-ae0dda82813d admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.065462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b08522-228c-4fda-a7c7-ae0dda82813d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.067360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b08522-228c-4fda-a7c7-ae0dda82813d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.095058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1042/2146] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:56:55 2022] POST /identity/v3/projects => generated 407 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:56:55.099375 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebc1c4-a02d-4368-943e-f3d2906828f1 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.110435 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebc1c4-a02d-4368-943e-f3d2906828f1 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.176121 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51617b-8ce1-49e8-ae82-8a8b980974f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.176976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51617b-8ce1-49e8-ae82-8a8b980974f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.177262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51617b-8ce1-49e8-ae82-8a8b980974f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.178177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51617b-8ce1-49e8-ae82-8a8b980974f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.178296 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51617b-8ce1-49e8-ae82-8a8b980974f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.180205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb51617b-8ce1-49e8-ae82-8a8b980974f8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.204427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1105/2147] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:56:55 2022] POST /identity/v3/users => generated 304 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:56:55.208171 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b08522-228c-4fda-a7c7-ae0dda82813d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.219557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b08522-228c-4fda-a7c7-ae0dda82813d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.225698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5e16c-cb9c-4eac-9fd6-e9f31c64848d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.226365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5e16c-cb9c-4eac-9fd6-e9f31c64848d admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.226787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5e16c-cb9c-4eac-9fd6-e9f31c64848d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.227069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed5e16c-cb9c-4eac-9fd6-e9f31c64848d admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.227805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5e16c-cb9c-4eac-9fd6-e9f31c64848d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.229629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed5e16c-cb9c-4eac-9fd6-e9f31c64848d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.241831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1043/2148] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:55.247538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51617b-8ce1-49e8-ae82-8a8b980974f8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.259850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51617b-8ce1-49e8-ae82-8a8b980974f8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.265856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74447c60-454c-45fd-8d96-c6a4d263b38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.266528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74447c60-454c-45fd-8d96-c6a4d263b38c admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.266821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74447c60-454c-45fd-8d96-c6a4d263b38c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.267105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74447c60-454c-45fd-8d96-c6a4d263b38c admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.267886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74447c60-454c-45fd-8d96-c6a4d263b38c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.269578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74447c60-454c-45fd-8d96-c6a4d263b38c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.280559 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1106/2149] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:55.284749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5e16c-cb9c-4eac-9fd6-e9f31c64848d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.296042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed5e16c-cb9c-4eac-9fd6-e9f31c64848d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.300645 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa29f7-71ab-459b-b931-0160c36b175f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.301546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa29f7-71ab-459b-b931-0160c36b175f admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.301828 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa29f7-71ab-459b-b931-0160c36b175f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.302104 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfa29f7-71ab-459b-b931-0160c36b175f admin admin] PATH_INFO: `/v3/projects/12f91a7d1aeb43158a70a6d69f6969b8/users/1e821ffa1d77475d95cf995d9c8d816c/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.316293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa29f7-71ab-459b-b931-0160c36b175f admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f91a7d1aeb43158a70a6d69f6969b8, user_id=1e821ffa1d77475d95cf995d9c8d816c, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.318964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfa29f7-71ab-459b-b931-0160c36b175f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.331803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1044/2150] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:55 2022] PUT /identity/v3/projects/12f91a7d1aeb43158a70a6d69f6969b8/users/1e821ffa1d77475d95cf995d9c8d816c/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:55.342866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363664e5-0b59-4c42-94e4-5748fdbf9793 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.343162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363664e5-0b59-4c42-94e4-5748fdbf9793 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.343473 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363664e5-0b59-4c42-94e4-5748fdbf9793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.372126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-363664e5-0b59-4c42-94e4-5748fdbf9793 None None] MFA Rules not processed for user `1e821ffa1d77475d95cf995d9c8d816c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:55.373399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-363664e5-0b59-4c42-94e4-5748fdbf9793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:55.415403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:55.439606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1107/2151] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:55 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:55.444088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa29f7-71ab-459b-b931-0160c36b175f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.492787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:55.511719 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfa29f7-71ab-459b-b931-0160c36b175f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.516637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e8257-56c9-476b-8174-9675064b180b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.517321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e8257-56c9-476b-8174-9675064b180b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.517605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e8257-56c9-476b-8174-9675064b180b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.517883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e8257-56c9-476b-8174-9675064b180b admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.518570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e8257-56c9-476b-8174-9675064b180b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.520528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e8257-56c9-476b-8174-9675064b180b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.547343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1045/2152] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:56:55 2022] POST /identity/v3/projects => generated 409 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:56:55.551443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363664e5-0b59-4c42-94e4-5748fdbf9793 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.562276 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363664e5-0b59-4c42-94e4-5748fdbf9793 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.622918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c791a-2da8-466f-a299-a9c7ac10d7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.623624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c791a-2da8-466f-a299-a9c7ac10d7cb admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.623936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c791a-2da8-466f-a299-a9c7ac10d7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.624216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c791a-2da8-466f-a299-a9c7ac10d7cb admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.625029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c791a-2da8-466f-a299-a9c7ac10d7cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.626908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c791a-2da8-466f-a299-a9c7ac10d7cb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.646862 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1108/2153] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:56:55 2022] POST /identity/v3/users => generated 311 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:56:55.651380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e8257-56c9-476b-8174-9675064b180b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.661986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e8257-56c9-476b-8174-9675064b180b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.667037 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8691657-a16d-4ae1-8d0d-6fb7830249d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.667728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8691657-a16d-4ae1-8d0d-6fb7830249d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.668010 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8691657-a16d-4ae1-8d0d-6fb7830249d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.668324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8691657-a16d-4ae1-8d0d-6fb7830249d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.669092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8691657-a16d-4ae1-8d0d-6fb7830249d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.670958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8691657-a16d-4ae1-8d0d-6fb7830249d1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.685645 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1046/2154] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:55.686437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c791a-2da8-466f-a299-a9c7ac10d7cb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.696352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c791a-2da8-466f-a299-a9c7ac10d7cb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.702177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db5324-3a1a-4321-b8a4-c3028b5f996e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.703071 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db5324-3a1a-4321-b8a4-c3028b5f996e admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.703354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db5324-3a1a-4321-b8a4-c3028b5f996e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.703627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db5324-3a1a-4321-b8a4-c3028b5f996e admin admin] PATH_INFO: `/v3/projects/da96fd5819bb42d58d39e2f6e8085a22/users/783a4e4869cf4cf3b896503bed2793da/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.720125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db5324-3a1a-4321-b8a4-c3028b5f996e admin admin] RBAC: Authorizing `identity:create_grant(project_id=da96fd5819bb42d58d39e2f6e8085a22, user_id=783a4e4869cf4cf3b896503bed2793da, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.722877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db5324-3a1a-4321-b8a4-c3028b5f996e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.736639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1109/2155] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:55 2022] PUT /identity/v3/projects/da96fd5819bb42d58d39e2f6e8085a22/users/783a4e4869cf4cf3b896503bed2793da/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:56:55.740661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8691657-a16d-4ae1-8d0d-6fb7830249d1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.751130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8691657-a16d-4ae1-8d0d-6fb7830249d1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.811123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c28f3-1223-4c53-870f-d9059a17af15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.811832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c28f3-1223-4c53-870f-d9059a17af15 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.812066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c28f3-1223-4c53-870f-d9059a17af15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.812343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c28f3-1223-4c53-870f-d9059a17af15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.813107 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c28f3-1223-4c53-870f-d9059a17af15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.814705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c28f3-1223-4c53-870f-d9059a17af15 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.828741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1047/2156] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:56:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:56:55.829982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db5324-3a1a-4321-b8a4-c3028b5f996e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.840446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db5324-3a1a-4321-b8a4-c3028b5f996e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:55.845225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12e987-25b7-4b0c-93e9-4602af404d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:55.846116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12e987-25b7-4b0c-93e9-4602af404d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.846404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12e987-25b7-4b0c-93e9-4602af404d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.846686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12e987-25b7-4b0c-93e9-4602af404d68 admin admin] PATH_INFO: `/v3/projects/da96fd5819bb42d58d39e2f6e8085a22/users/783a4e4869cf4cf3b896503bed2793da/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.849857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12e987-25b7-4b0c-93e9-4602af404d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da96fd5819bb42d58d39e2f6e8085a22, user_id=783a4e4869cf4cf3b896503bed2793da, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:55.852500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12e987-25b7-4b0c-93e9-4602af404d68 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:55.867407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1110/2157] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:56:55 2022] PUT /identity/v3/projects/da96fd5819bb42d58d39e2f6e8085a22/users/783a4e4869cf4cf3b896503bed2793da/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:56:55.878288 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a73b4-fac4-4bc2-8a72-5dc889fdb361 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:55.878662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a73b4-fac4-4bc2-8a72-5dc889fdb361 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:55.879023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a73b4-fac4-4bc2-8a72-5dc889fdb361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:55.905436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ca0a73b4-fac4-4bc2-8a72-5dc889fdb361 None None] MFA Rules not processed for user `783a4e4869cf4cf3b896503bed2793da`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:56:55.906687 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ca0a73b4-fac4-4bc2-8a72-5dc889fdb361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:55.974569 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1048/2158] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:56:55 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:56:55.985592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12e987-25b7-4b0c-93e9-4602af404d68 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:55.987336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7b12e987-25b7-4b0c-93e9-4602af404d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:56:56.046980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12e987-25b7-4b0c-93e9-4602af404d68 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:56.050586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc82be-5655-4b8a-83de-1a7d9c4e648a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:56.051236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc82be-5655-4b8a-83de-1a7d9c4e648a service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:56.051530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc82be-5655-4b8a-83de-1a7d9c4e648a service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:56.051810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc82be-5655-4b8a-83de-1a7d9c4e648a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:56.062395 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc82be-5655-4b8a-83de-1a7d9c4e648a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:56.063858 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc82be-5655-4b8a-83de-1a7d9c4e648a service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:56.073944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1111/2159] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:55 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:56:56.228622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a73b4-fac4-4bc2-8a72-5dc889fdb361 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:56:56.238894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a73b4-fac4-4bc2-8a72-5dc889fdb361 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:56:56.242468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c624133-762a-4f1d-8960-41f0843cca48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:56:56.243101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c624133-762a-4f1d-8960-41f0843cca48 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:56:56.243387 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c624133-762a-4f1d-8960-41f0843cca48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:56:56.243663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c624133-762a-4f1d-8960-41f0843cca48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:56:56.279553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:56:56.306966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c624133-762a-4f1d-8960-41f0843cca48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:56:56.309083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c624133-762a-4f1d-8960-41f0843cca48 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:56:56.318648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1049/2160] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:56:56 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:05.734749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc82be-5655-4b8a-83de-1a7d9c4e648a service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:05.785278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:05.818399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc82be-5655-4b8a-83de-1a7d9c4e648a service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:05.826694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:05.828214 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.828513 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.828827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] PATH_INFO: `/v3/users/1e821ffa1d77475d95cf995d9c8d816c` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.830471 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e821ffa1d77475d95cf995d9c8d816c)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:05.835506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:05.892217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e821ffa1d77475d95cf995d9c8d816c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:05.892908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e821ffa1d77475d95cf995d9c8d816c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:05.893553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e821ffa1d77475d95cf995d9c8d816c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:05.907167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e821ffa1d77475d95cf995d9c8d816c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:05.914490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e821ffa1d77475d95cf995d9c8d816c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:05.928008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1112/2161] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:05 2022] DELETE /identity/v3/users/1e821ffa1d77475d95cf995d9c8d816c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:05.933635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47729442-299f-437b-9b16-88f4aef2f0f6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:05.934093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47729442-299f-437b-9b16-88f4aef2f0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:05.934299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47729442-299f-437b-9b16-88f4aef2f0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:05.993529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-47729442-299f-437b-9b16-88f4aef2f0f6 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:05.995386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-47729442-299f-437b-9b16-88f4aef2f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:06.093442 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1050/2162] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:06.102844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:06.104863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:06.206030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bee72a-3ed6-4218-aabc-0537a07a1028 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:06.219845 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c2403-5c4e-4a51-a6f8-de46f5b978e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:06.220573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c2403-5c4e-4a51-a6f8-de46f5b978e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:06.221425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c2403-5c4e-4a51-a6f8-de46f5b978e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:06.221425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316c2403-5c4e-4a51-a6f8-de46f5b978e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:06.232160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c2403-5c4e-4a51-a6f8-de46f5b978e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:06.233859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316c2403-5c4e-4a51-a6f8-de46f5b978e8 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:06.245456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1113/2163] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:06.861526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47729442-299f-437b-9b16-88f4aef2f0f6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:06.864380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-47729442-299f-437b-9b16-88f4aef2f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:06.888760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47729442-299f-437b-9b16-88f4aef2f0f6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:06.895442 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:06.896597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:06.897103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:06.897576 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] PATH_INFO: `/v3/projects/12f91a7d1aeb43158a70a6d69f6969b8` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:06.899123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] RBAC: Authorizing `identity:delete_project(project_id=12f91a7d1aeb43158a70a6d69f6969b8)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:06.901890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:06.956442 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12f91a7d1aeb43158a70a6d69f6969b8', 'request_id': 'req-7b226d8f-2387-46f7-b4be-e3144e91760a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:06.965437 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12f91a7d1aeb43158a70a6d69f6969b8', 'request_id': 'req-7b226d8f-2387-46f7-b4be-e3144e91760a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:06.972169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12f91a7d1aeb43158a70a6d69f6969b8', 'request_id': 'req-7b226d8f-2387-46f7-b4be-e3144e91760a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:06.974316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:06.985250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] The token cache is being invalidate because project 12f91a7d1aeb43158a70a6d69f6969b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:06.985622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:06.986365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:06.987985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1051/2164] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:06 2022] DELETE /identity/v3/projects/12f91a7d1aeb43158a70a6d69f6969b8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:06.991912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c2403-5c4e-4a51-a6f8-de46f5b978e8 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:07.002098 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316c2403-5c4e-4a51-a6f8-de46f5b978e8 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:07.019798 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:07.060889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b2efd-cc93-4209-9a8d-783e16937d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:07.061603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b2efd-cc93-4209-9a8d-783e16937d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.061891 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b2efd-cc93-4209-9a8d-783e16937d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.062227 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b2efd-cc93-4209-9a8d-783e16937d6e admin admin] PATH_INFO: `/v3/users/783a4e4869cf4cf3b896503bed2793da` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.063428 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b2efd-cc93-4209-9a8d-783e16937d6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=783a4e4869cf4cf3b896503bed2793da)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:07.065392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b2efd-cc93-4209-9a8d-783e16937d6e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:07.110413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-515b2efd-cc93-4209-9a8d-783e16937d6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '783a4e4869cf4cf3b896503bed2793da'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.111065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-515b2efd-cc93-4209-9a8d-783e16937d6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '783a4e4869cf4cf3b896503bed2793da'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.111679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-515b2efd-cc93-4209-9a8d-783e16937d6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '783a4e4869cf4cf3b896503bed2793da'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.124482 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-515b2efd-cc93-4209-9a8d-783e16937d6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '783a4e4869cf4cf3b896503bed2793da'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.131445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-515b2efd-cc93-4209-9a8d-783e16937d6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '783a4e4869cf4cf3b896503bed2793da'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.146668 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1114/2165] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:06 2022] DELETE /identity/v3/users/783a4e4869cf4cf3b896503bed2793da => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:07.413631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:07.415444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:07.430082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b226d8f-2387-46f7-b4be-e3144e91760a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:07.487813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6526e5-7fbb-4295-82f2-633072417e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:07.488699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6526e5-7fbb-4295-82f2-633072417e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.488987 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6526e5-7fbb-4295-82f2-633072417e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.489290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6526e5-7fbb-4295-82f2-633072417e68 admin admin] PATH_INFO: `/v3/projects/da96fd5819bb42d58d39e2f6e8085a22` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.490469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6526e5-7fbb-4295-82f2-633072417e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da96fd5819bb42d58d39e2f6e8085a22)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:07.492384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6526e5-7fbb-4295-82f2-633072417e68 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:07.540380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0a6526e5-7fbb-4295-82f2-633072417e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da96fd5819bb42d58d39e2f6e8085a22', 'request_id': 'req-0a6526e5-7fbb-4295-82f2-633072417e68'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.549726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0a6526e5-7fbb-4295-82f2-633072417e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da96fd5819bb42d58d39e2f6e8085a22', 'request_id': 'req-0a6526e5-7fbb-4295-82f2-633072417e68'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.559304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0a6526e5-7fbb-4295-82f2-633072417e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da96fd5819bb42d58d39e2f6e8085a22', 'request_id': 'req-0a6526e5-7fbb-4295-82f2-633072417e68'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.574024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0a6526e5-7fbb-4295-82f2-633072417e68 admin admin] The token cache is being invalidate because project da96fd5819bb42d58d39e2f6e8085a22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:07.574392 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0a6526e5-7fbb-4295-82f2-633072417e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.575165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-0a6526e5-7fbb-4295-82f2-633072417e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:07.576964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1052/2166] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:07 2022] DELETE /identity/v3/projects/da96fd5819bb42d58d39e2f6e8085a22 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:07.586254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eb004-8893-4841-8035-38d3eaad5079 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.586671 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eb004-8893-4841-8035-38d3eaad5079 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.586961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000eb004-8893-4841-8035-38d3eaad5079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.630010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-000eb004-8893-4841-8035-38d3eaad5079 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:07.631541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-000eb004-8893-4841-8035-38d3eaad5079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:07.721814 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1115/2167] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:07.730554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d9b69-49a3-48c1-8460-13f066160b2c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.730848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d9b69-49a3-48c1-8460-13f066160b2c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.731280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d9b69-49a3-48c1-8460-13f066160b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.750286 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1b1d9b69-49a3-48c1-8460-13f066160b2c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:07.751504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1b1d9b69-49a3-48c1-8460-13f066160b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:07.759712 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1053/2168] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:07.764717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000eb004-8893-4841-8035-38d3eaad5079 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:07.774785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000eb004-8893-4841-8035-38d3eaad5079 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:07.779952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c9c39-a2a2-4ca1-8366-5a8b1caeab3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:07.780607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c9c39-a2a2-4ca1-8366-5a8b1caeab3f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.780927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c9c39-a2a2-4ca1-8366-5a8b1caeab3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.781206 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c9c39-a2a2-4ca1-8366-5a8b1caeab3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.781944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c9c39-a2a2-4ca1-8366-5a8b1caeab3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:07.782250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c9c39-a2a2-4ca1-8366-5a8b1caeab3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:07.784022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c9c39-a2a2-4ca1-8366-5a8b1caeab3f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:07.797585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1116/2169] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:57:07.802653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d9b69-49a3-48c1-8460-13f066160b2c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:07.814949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d9b69-49a3-48c1-8460-13f066160b2c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:07.819893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f9d73-ad08-4011-8a16-8ae2ae804b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:07.820798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f9d73-ad08-4011-8a16-8ae2ae804b19 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.821071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f9d73-ad08-4011-8a16-8ae2ae804b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.821355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f9d73-ad08-4011-8a16-8ae2ae804b19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.822064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f9d73-ad08-4011-8a16-8ae2ae804b19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:07.824759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f9d73-ad08-4011-8a16-8ae2ae804b19 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:07.861248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1054/2170] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:07 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:07.866113 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c9c39-a2a2-4ca1-8366-5a8b1caeab3f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:07.879037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c9c39-a2a2-4ca1-8366-5a8b1caeab3f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:07.894263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:07.948882 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d5611-8f26-4ccc-a849-cf7f21538983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:07.949669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d5611-8f26-4ccc-a849-cf7f21538983 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.949959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d5611-8f26-4ccc-a849-cf7f21538983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.950246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d5611-8f26-4ccc-a849-cf7f21538983 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.950999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d5611-8f26-4ccc-a849-cf7f21538983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:07.953117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d5611-8f26-4ccc-a849-cf7f21538983 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:07.975670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1117/2171] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:07 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:07.979550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f9d73-ad08-4011-8a16-8ae2ae804b19 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:07.991129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f9d73-ad08-4011-8a16-8ae2ae804b19 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:07.996855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce101a-9e18-4be6-ad80-4460d4efaef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:07.997693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce101a-9e18-4be6-ad80-4460d4efaef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:07.998079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce101a-9e18-4be6-ad80-4460d4efaef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:07.998467 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce101a-9e18-4be6-ad80-4460d4efaef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:07.999321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce101a-9e18-4be6-ad80-4460d4efaef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:08.001513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce101a-9e18-4be6-ad80-4460d4efaef5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:08.028579 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1055/2172] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:08.029923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d5611-8f26-4ccc-a849-cf7f21538983 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:08.040216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d5611-8f26-4ccc-a849-cf7f21538983 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:08.045828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202dd38d-4084-4d6a-8de7-de253ed3821f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:08.046520 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dd38d-4084-4d6a-8de7-de253ed3821f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:08.046809 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dd38d-4084-4d6a-8de7-de253ed3821f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:08.047094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dd38d-4084-4d6a-8de7-de253ed3821f admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:08.047849 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202dd38d-4084-4d6a-8de7-de253ed3821f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:08.049512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202dd38d-4084-4d6a-8de7-de253ed3821f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:08.060618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1118/2173] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:08.065022 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce101a-9e18-4be6-ad80-4460d4efaef5 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:08.076101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce101a-9e18-4be6-ad80-4460d4efaef5 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:08.080950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dba0f8-0dc4-4ef1-b0cc-b965eb24ec9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:08.081838 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dba0f8-0dc4-4ef1-b0cc-b965eb24ec9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:08.082121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dba0f8-0dc4-4ef1-b0cc-b965eb24ec9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:08.082421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dba0f8-0dc4-4ef1-b0cc-b965eb24ec9f admin admin] PATH_INFO: `/v3/projects/d99d98fb29fb4f44aac82291b4e6f370/users/1a085194a93f46ecaf0cd9273fe26e90/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:08.097457 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dba0f8-0dc4-4ef1-b0cc-b965eb24ec9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d99d98fb29fb4f44aac82291b4e6f370, user_id=1a085194a93f46ecaf0cd9273fe26e90, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:08.100047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dba0f8-0dc4-4ef1-b0cc-b965eb24ec9f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:08.113747 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1056/2174] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:08 2022] PUT /identity/v3/projects/d99d98fb29fb4f44aac82291b4e6f370/users/1a085194a93f46ecaf0cd9273fe26e90/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:08.124877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42dcea-809a-4bdc-b229-611eaedda8da None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:08.125175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42dcea-809a-4bdc-b229-611eaedda8da None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:08.125459 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42dcea-809a-4bdc-b229-611eaedda8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:08.153250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0e42dcea-809a-4bdc-b229-611eaedda8da None None] MFA Rules not processed for user `1a085194a93f46ecaf0cd9273fe26e90`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:08.154484 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0e42dcea-809a-4bdc-b229-611eaedda8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:08.217263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1119/2175] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:08.228588 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dba0f8-0dc4-4ef1-b0cc-b965eb24ec9f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:08.230895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-96dba0f8-0dc4-4ef1-b0cc-b965eb24ec9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:08.251562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:08.297670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dba0f8-0dc4-4ef1-b0cc-b965eb24ec9f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:08.301319 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5056a71-a55c-49cb-8e4b-8a4046bdf38f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:08.301957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5056a71-a55c-49cb-8e4b-8a4046bdf38f service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:08.302246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5056a71-a55c-49cb-8e4b-8a4046bdf38f service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:08.302523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5056a71-a55c-49cb-8e4b-8a4046bdf38f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:08.311409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5056a71-a55c-49cb-8e4b-8a4046bdf38f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:08.312871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5056a71-a55c-49cb-8e4b-8a4046bdf38f service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:08.321845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1057/2176] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:09.880895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42dcea-809a-4bdc-b229-611eaedda8da None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:09.882660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0e42dcea-809a-4bdc-b229-611eaedda8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:09.950376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42dcea-809a-4bdc-b229-611eaedda8da None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:09.955012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:09.955886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:09.956171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:09.956566 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] PATH_INFO: `/v3/users/d617fda3ed0a4a11a90bcae8b714dfab` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:09.957684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=d617fda3ed0a4a11a90bcae8b714dfab)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:09.959634 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:09.981530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:10.009638 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd617fda3ed0a4a11a90bcae8b714dfab'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.010492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd617fda3ed0a4a11a90bcae8b714dfab'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.011128 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd617fda3ed0a4a11a90bcae8b714dfab'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.024017 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd617fda3ed0a4a11a90bcae8b714dfab'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.031449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd617fda3ed0a4a11a90bcae8b714dfab'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.044613 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1120/2177] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:09 2022] DELETE /identity/v3/users/d617fda3ed0a4a11a90bcae8b714dfab => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:10.049406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52979787-6b83-4520-bc8f-630059125158 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.049863 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52979787-6b83-4520-bc8f-630059125158 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.050035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52979787-6b83-4520-bc8f-630059125158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.068277 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-52979787-6b83-4520-bc8f-630059125158 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:10.069682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-52979787-6b83-4520-bc8f-630059125158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.133482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1058/2178] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:10.143018 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.144450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.208460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6f383e-118a-4e9f-b17d-4aeac9e045ca admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.214309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cc8ac-9a15-4923-8d4e-b9348a9b266b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.215047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cc8ac-9a15-4923-8d4e-b9348a9b266b service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.215409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cc8ac-9a15-4923-8d4e-b9348a9b266b service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.215860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cc8ac-9a15-4923-8d4e-b9348a9b266b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.226755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732cc8ac-9a15-4923-8d4e-b9348a9b266b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.230525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732cc8ac-9a15-4923-8d4e-b9348a9b266b service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.255409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1121/2179] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:10.636439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52979787-6b83-4520-bc8f-630059125158 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.638745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-52979787-6b83-4520-bc8f-630059125158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.650800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52979787-6b83-4520-bc8f-630059125158 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.656524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.657365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.657659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.657944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f admin admin] PATH_INFO: `/v3/projects/67adea7232d041a19014f0a75c361a18` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.659174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67adea7232d041a19014f0a75c361a18)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.661184 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.710788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67adea7232d041a19014f0a75c361a18', 'request_id': 'req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.719658 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67adea7232d041a19014f0a75c361a18', 'request_id': 'req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.726004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67adea7232d041a19014f0a75c361a18', 'request_id': 'req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.736407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f admin admin] The token cache is being invalidate because project 67adea7232d041a19014f0a75c361a18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:10.736725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.737378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4ed942e4-4ac2-4f52-acf9-c6765cc7ce8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.738919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1059/2180] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:10 2022] DELETE /identity/v3/projects/67adea7232d041a19014f0a75c361a18 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:10.743735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b456db-b357-4895-a191-7254dc9b0b7e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.744053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b456db-b357-4895-a191-7254dc9b0b7e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.744340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b456db-b357-4895-a191-7254dc9b0b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.762715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-99b456db-b357-4895-a191-7254dc9b0b7e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:10.763934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-99b456db-b357-4895-a191-7254dc9b0b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.822524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1122/2181] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:10.831027 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d21851-bc0e-4ed9-bbbd-14db68a57bf4 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.831323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d21851-bc0e-4ed9-bbbd-14db68a57bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.831783 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d21851-bc0e-4ed9-bbbd-14db68a57bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.849035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b456db-b357-4895-a191-7254dc9b0b7e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.850413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-99b456db-b357-4895-a191-7254dc9b0b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.850831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-25d21851-bc0e-4ed9-bbbd-14db68a57bf4 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:10.852019 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-25d21851-bc0e-4ed9-bbbd-14db68a57bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.860841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1060/2182] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:10.862685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b456db-b357-4895-a191-7254dc9b0b7e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.865140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d21851-bc0e-4ed9-bbbd-14db68a57bf4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.867261 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc0f55-9b40-450c-9f95-d88781ac10bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.868108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0f55-9b40-450c-9f95-d88781ac10bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.868393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0f55-9b40-450c-9f95-d88781ac10bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.868693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0f55-9b40-450c-9f95-d88781ac10bf admin admin] PATH_INFO: `/v3/users/1a085194a93f46ecaf0cd9273fe26e90` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.869904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc0f55-9b40-450c-9f95-d88781ac10bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a085194a93f46ecaf0cd9273fe26e90)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.871819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc0f55-9b40-450c-9f95-d88781ac10bf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.875633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d21851-bc0e-4ed9-bbbd-14db68a57bf4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.881131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7bec9-790d-435f-976b-21d26d01f4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.881805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7bec9-790d-435f-976b-21d26d01f4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.882090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7bec9-790d-435f-976b-21d26d01f4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.882368 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7bec9-790d-435f-976b-21d26d01f4a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.883093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7bec9-790d-435f-976b-21d26d01f4a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:10.883401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7bec9-790d-435f-976b-21d26d01f4a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.885071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7bec9-790d-435f-976b-21d26d01f4a5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.901906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1061/2183] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 11:57:10.902941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7bec9-790d-435f-976b-21d26d01f4a5 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.914516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:10.915516 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7bec9-790d-435f-976b-21d26d01f4a5 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:10.922268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ebb52-5075-43bb-8008-d0410f06c933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:10.922938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ebb52-5075-43bb-8008-d0410f06c933 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.923236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ebb52-5075-43bb-8008-d0410f06c933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.923522 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ebb52-5075-43bb-8008-d0410f06c933 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.924427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ebb52-5075-43bb-8008-d0410f06c933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:10.926467 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426ebb52-5075-43bb-8008-d0410f06c933 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:10.932187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d4fc0f55-9b40-450c-9f95-d88781ac10bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a085194a93f46ecaf0cd9273fe26e90'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.932906 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d4fc0f55-9b40-450c-9f95-d88781ac10bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a085194a93f46ecaf0cd9273fe26e90'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.933568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d4fc0f55-9b40-450c-9f95-d88781ac10bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a085194a93f46ecaf0cd9273fe26e90'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.946530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d4fc0f55-9b40-450c-9f95-d88781ac10bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a085194a93f46ecaf0cd9273fe26e90'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.953880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d4fc0f55-9b40-450c-9f95-d88781ac10bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a085194a93f46ecaf0cd9273fe26e90'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:10.957947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1062/2184] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:10 2022] POST /identity/v3/projects => generated 395 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:10.959484 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ebb52-5075-43bb-8008-d0410f06c933 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:10.961490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-426ebb52-5075-43bb-8008-d0410f06c933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:10.966909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:10.968783 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1123/2185] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:10 2022] DELETE /identity/v3/users/1a085194a93f46ecaf0cd9273fe26e90 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:57:10.974740 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7635e-0f5e-4b6c-a116-cbd38211269b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:10.975095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7635e-0f5e-4b6c-a116-cbd38211269b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:10.975875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7635e-0f5e-4b6c-a116-cbd38211269b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:10.994515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e7a7635e-0f5e-4b6c-a116-cbd38211269b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:10.995765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e7a7635e-0f5e-4b6c-a116-cbd38211269b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:11.035728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ebb52-5075-43bb-8008-d0410f06c933 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.069473 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1124/2186] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:11.079688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7635e-0f5e-4b6c-a116-cbd38211269b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.081330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e7a7635e-0f5e-4b6c-a116-cbd38211269b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:11.099293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25139853-cdb9-4b73-b7ce-fa10660eeab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.100094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25139853-cdb9-4b73-b7ce-fa10660eeab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.100380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25139853-cdb9-4b73-b7ce-fa10660eeab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.100660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25139853-cdb9-4b73-b7ce-fa10660eeab8 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.101389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25139853-cdb9-4b73-b7ce-fa10660eeab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.103375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25139853-cdb9-4b73-b7ce-fa10660eeab8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.125227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1063/2187] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:10 2022] POST /identity/v3/users => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:11.129152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25139853-cdb9-4b73-b7ce-fa10660eeab8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.139486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7635e-0f5e-4b6c-a116-cbd38211269b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.140275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25139853-cdb9-4b73-b7ce-fa10660eeab8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.143216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef572cf-919a-4f5b-bdfb-26473214f99b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.143874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef572cf-919a-4f5b-bdfb-26473214f99b service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.144165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef572cf-919a-4f5b-bdfb-26473214f99b service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.144448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef572cf-919a-4f5b-bdfb-26473214f99b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.145180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc8484-c9db-4790-8eee-9ea19690e4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.145842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc8484-c9db-4790-8eee-9ea19690e4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.146131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc8484-c9db-4790-8eee-9ea19690e4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.146527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc8484-c9db-4790-8eee-9ea19690e4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.147269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc8484-c9db-4790-8eee-9ea19690e4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.148934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc8484-c9db-4790-8eee-9ea19690e4f6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.153919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef572cf-919a-4f5b-bdfb-26473214f99b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.155359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef572cf-919a-4f5b-bdfb-26473214f99b service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.159381 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1064/2188] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:11.163170 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc8484-c9db-4790-8eee-9ea19690e4f6 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.165730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1125/2189] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:57:11.173482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc8484-c9db-4790-8eee-9ea19690e4f6 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.178233 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc44cb9-93ba-4e57-a232-1f6eb0151276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.178887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc44cb9-93ba-4e57-a232-1f6eb0151276 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.179180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc44cb9-93ba-4e57-a232-1f6eb0151276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.179555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc44cb9-93ba-4e57-a232-1f6eb0151276 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.180181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc44cb9-93ba-4e57-a232-1f6eb0151276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.181895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc44cb9-93ba-4e57-a232-1f6eb0151276 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.193248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1065/2190] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:11.197680 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef572cf-919a-4f5b-bdfb-26473214f99b service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.208565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef572cf-919a-4f5b-bdfb-26473214f99b service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.214438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb14d0-9ea0-4f4f-87bd-261385d271bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.215490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb14d0-9ea0-4f4f-87bd-261385d271bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.215776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb14d0-9ea0-4f4f-87bd-261385d271bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.216050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb14d0-9ea0-4f4f-87bd-261385d271bc admin admin] PATH_INFO: `/v3/projects/a320d40801364c9f9843446a627dd421/users/319f0c90144941f792d676391e9889e4/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.233136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb14d0-9ea0-4f4f-87bd-261385d271bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a320d40801364c9f9843446a627dd421, user_id=319f0c90144941f792d676391e9889e4, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.236125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb14d0-9ea0-4f4f-87bd-261385d271bc admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.252051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1126/2191] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:11 2022] PUT /identity/v3/projects/a320d40801364c9f9843446a627dd421/users/319f0c90144941f792d676391e9889e4/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:11.262969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc7d6a-21e9-4a34-84af-fb3fa5ae01e4 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.263262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc7d6a-21e9-4a34-84af-fb3fa5ae01e4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.263544 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc7d6a-21e9-4a34-84af-fb3fa5ae01e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.289801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-96cc7d6a-21e9-4a34-84af-fb3fa5ae01e4 None None] MFA Rules not processed for user `319f0c90144941f792d676391e9889e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:11.291023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-96cc7d6a-21e9-4a34-84af-fb3fa5ae01e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:11.368172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1066/2192] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:11.380285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb14d0-9ea0-4f4f-87bd-261385d271bc admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.382204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-72eb14d0-9ea0-4f4f-87bd-261385d271bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:11.423701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:11.459682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc7d6a-21e9-4a34-84af-fb3fa5ae01e4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:11.461361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-96cc7d6a-21e9-4a34-84af-fb3fa5ae01e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:11.474578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb14d0-9ea0-4f4f-87bd-261385d271bc admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.478423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:11.483689 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dbbfc-f7ec-4842-b503-b8b96b51968c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.486340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dbbfc-f7ec-4842-b503-b8b96b51968c service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.486700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dbbfc-f7ec-4842-b503-b8b96b51968c service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.487044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dbbfc-f7ec-4842-b503-b8b96b51968c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.511499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133dbbfc-f7ec-4842-b503-b8b96b51968c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.513152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133dbbfc-f7ec-4842-b503-b8b96b51968c service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.527445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1127/2193] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:11.557838 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc7d6a-21e9-4a34-84af-fb3fa5ae01e4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:11.571958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848cc8e-81b4-4b51-ad30-996ea204d9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:11.572915 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848cc8e-81b4-4b51-ad30-996ea204d9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.573283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848cc8e-81b4-4b51-ad30-996ea204d9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.573474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848cc8e-81b4-4b51-ad30-996ea204d9e7 admin admin] PATH_INFO: `/v3/projects/d99d98fb29fb4f44aac82291b4e6f370` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.574751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848cc8e-81b4-4b51-ad30-996ea204d9e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d99d98fb29fb4f44aac82291b4e6f370)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:11.577739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848cc8e-81b4-4b51-ad30-996ea204d9e7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:11.660990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9848cc8e-81b4-4b51-ad30-996ea204d9e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd99d98fb29fb4f44aac82291b4e6f370', 'request_id': 'req-9848cc8e-81b4-4b51-ad30-996ea204d9e7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:11.675793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9848cc8e-81b4-4b51-ad30-996ea204d9e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd99d98fb29fb4f44aac82291b4e6f370', 'request_id': 'req-9848cc8e-81b4-4b51-ad30-996ea204d9e7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:11.684339 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9848cc8e-81b4-4b51-ad30-996ea204d9e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd99d98fb29fb4f44aac82291b4e6f370', 'request_id': 'req-9848cc8e-81b4-4b51-ad30-996ea204d9e7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:11.710825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9848cc8e-81b4-4b51-ad30-996ea204d9e7 admin admin] The token cache is being invalidate because project d99d98fb29fb4f44aac82291b4e6f370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:11.711218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9848cc8e-81b4-4b51-ad30-996ea204d9e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:11.717399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9848cc8e-81b4-4b51-ad30-996ea204d9e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:11.719730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1067/2194] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:11 2022] DELETE /identity/v3/projects/d99d98fb29fb4f44aac82291b4e6f370 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:11.726724 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70732fc0-7b87-49a3-8548-917fe9e959f2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.727011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70732fc0-7b87-49a3-8548-917fe9e959f2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.727298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70732fc0-7b87-49a3-8548-917fe9e959f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.748866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-70732fc0-7b87-49a3-8548-917fe9e959f2 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:11.750569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-70732fc0-7b87-49a3-8548-917fe9e959f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:11.863293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1128/2195] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:11.881899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e1409-0c3a-4b06-8054-127eb77f3eaf None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:11.882539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e1409-0c3a-4b06-8054-127eb77f3eaf None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:11.883203 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e1409-0c3a-4b06-8054-127eb77f3eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:11.947725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8b8e1409-0c3a-4b06-8054-127eb77f3eaf None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:11.955308 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8b8e1409-0c3a-4b06-8054-127eb77f3eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:11.977801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1068/2196] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:11.983925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70732fc0-7b87-49a3-8548-917fe9e959f2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.009647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70732fc0-7b87-49a3-8548-917fe9e959f2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.024398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bf925-eb9d-4a27-a8c3-3f87f5dc5d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.026060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bf925-eb9d-4a27-a8c3-3f87f5dc5d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.026361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bf925-eb9d-4a27-a8c3-3f87f5dc5d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.026630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3bf925-eb9d-4a27-a8c3-3f87f5dc5d76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.027418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bf925-eb9d-4a27-a8c3-3f87f5dc5d76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:12.027732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bf925-eb9d-4a27-a8c3-3f87f5dc5d76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.034034 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3bf925-eb9d-4a27-a8c3-3f87f5dc5d76 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.049776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1129/2197] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:57:12.056461 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e1409-0c3a-4b06-8054-127eb77f3eaf None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.072184 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e1409-0c3a-4b06-8054-127eb77f3eaf None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.086193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2942cb-f729-4947-b4d9-7ae936ba9365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.086928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2942cb-f729-4947-b4d9-7ae936ba9365 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.087245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2942cb-f729-4947-b4d9-7ae936ba9365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.087506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2942cb-f729-4947-b4d9-7ae936ba9365 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.088330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2942cb-f729-4947-b4d9-7ae936ba9365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.090681 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2942cb-f729-4947-b4d9-7ae936ba9365 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.124553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1069/2198] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:12 2022] POST /identity/v3/projects => generated 375 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:12.130532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bf925-eb9d-4a27-a8c3-3f87f5dc5d76 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.144846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3bf925-eb9d-4a27-a8c3-3f87f5dc5d76 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.253875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812536c-fd4d-41a2-b1de-e2e6b8c8eeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.254641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812536c-fd4d-41a2-b1de-e2e6b8c8eeaa admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.254929 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812536c-fd4d-41a2-b1de-e2e6b8c8eeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.255216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812536c-fd4d-41a2-b1de-e2e6b8c8eeaa admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.255945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812536c-fd4d-41a2-b1de-e2e6b8c8eeaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.257934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812536c-fd4d-41a2-b1de-e2e6b8c8eeaa admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.282992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1130/2199] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:12 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:12.287516 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2942cb-f729-4947-b4d9-7ae936ba9365 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.299771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2942cb-f729-4947-b4d9-7ae936ba9365 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.305417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2347d-dd2e-40b7-be3a-eaf678f84f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.306209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2347d-dd2e-40b7-be3a-eaf678f84f0c admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.306470 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2347d-dd2e-40b7-be3a-eaf678f84f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.306770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2347d-dd2e-40b7-be3a-eaf678f84f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.308043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2347d-dd2e-40b7-be3a-eaf678f84f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.309776 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2347d-dd2e-40b7-be3a-eaf678f84f0c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.325811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1070/2200] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:12.329501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812536c-fd4d-41a2-b1de-e2e6b8c8eeaa admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.340332 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812536c-fd4d-41a2-b1de-e2e6b8c8eeaa admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.345192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ba1ba7-a0b9-4ecf-8c57-67350dd5a63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.345947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba1ba7-a0b9-4ecf-8c57-67350dd5a63e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.346242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba1ba7-a0b9-4ecf-8c57-67350dd5a63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.346524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba1ba7-a0b9-4ecf-8c57-67350dd5a63e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.347280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba1ba7-a0b9-4ecf-8c57-67350dd5a63e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.349006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ba1ba7-a0b9-4ecf-8c57-67350dd5a63e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.360097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1131/2201] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:12.364600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2347d-dd2e-40b7-be3a-eaf678f84f0c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.374980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2347d-dd2e-40b7-be3a-eaf678f84f0c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.379654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584a320-ec30-4582-9853-1c4fcd13d0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.380525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584a320-ec30-4582-9853-1c4fcd13d0b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.380847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584a320-ec30-4582-9853-1c4fcd13d0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.381124 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584a320-ec30-4582-9853-1c4fcd13d0b3 admin admin] PATH_INFO: `/v3/projects/5dac2d4d38da494a8cd4cd136fe54f5b/users/158ea46d74bc4bcf95f81e0c6c878e88/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.396990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584a320-ec30-4582-9853-1c4fcd13d0b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dac2d4d38da494a8cd4cd136fe54f5b, user_id=158ea46d74bc4bcf95f81e0c6c878e88, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.399713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e584a320-ec30-4582-9853-1c4fcd13d0b3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.415374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1071/2202] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:12 2022] PUT /identity/v3/projects/5dac2d4d38da494a8cd4cd136fe54f5b/users/158ea46d74bc4bcf95f81e0c6c878e88/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:12.426636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a9f40-55c2-42a6-ad53-ae429405664a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.426933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a9f40-55c2-42a6-ad53-ae429405664a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.427220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a9f40-55c2-42a6-ad53-ae429405664a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.455781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b63a9f40-55c2-42a6-ad53-ae429405664a None None] MFA Rules not processed for user `158ea46d74bc4bcf95f81e0c6c878e88`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:12.457097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b63a9f40-55c2-42a6-ad53-ae429405664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:12.491183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:12.532251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1132/2203] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:12.545015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584a320-ec30-4582-9853-1c4fcd13d0b3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:12.547651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e584a320-ec30-4582-9853-1c4fcd13d0b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:12.584950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:12.611540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584a320-ec30-4582-9853-1c4fcd13d0b3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:12.616195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80302ff8-27e9-44db-bf7b-96f06f6ee50a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:12.616873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80302ff8-27e9-44db-bf7b-96f06f6ee50a service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:12.617292 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80302ff8-27e9-44db-bf7b-96f06f6ee50a service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:12.617575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80302ff8-27e9-44db-bf7b-96f06f6ee50a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:12.626870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80302ff8-27e9-44db-bf7b-96f06f6ee50a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:12.628326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80302ff8-27e9-44db-bf7b-96f06f6ee50a service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:12.638750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1072/2204] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:20.219576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a9f40-55c2-42a6-ad53-ae429405664a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:20.226983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b63a9f40-55c2-42a6-ad53-ae429405664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:20.320462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a9f40-55c2-42a6-ad53-ae429405664a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.325174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.326091 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.326514 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.326800 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] PATH_INFO: `/v3/users/319f0c90144941f792d676391e9889e4` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.328256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=319f0c90144941f792d676391e9889e4)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.330297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.389179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '319f0c90144941f792d676391e9889e4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:20.390354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '319f0c90144941f792d676391e9889e4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:20.391434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '319f0c90144941f792d676391e9889e4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:20.429872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '319f0c90144941f792d676391e9889e4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:20.445763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '319f0c90144941f792d676391e9889e4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:20.486718 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1133/2205] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:20 2022] DELETE /identity/v3/users/319f0c90144941f792d676391e9889e4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:20.489366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157204f-f5c7-40dc-85a1-934b787b9f68 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.490098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157204f-f5c7-40dc-85a1-934b787b9f68 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.490098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9157204f-f5c7-40dc-85a1-934b787b9f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.511870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9157204f-f5c7-40dc-85a1-934b787b9f68 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:20.513194 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9157204f-f5c7-40dc-85a1-934b787b9f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:20.615911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1073/2206] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:57:20.625787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:20.627330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:20.713879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49559e75-0fa8-4301-8dd8-f0993f8adf1a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:20.719285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bfe7f7-7a5b-4677-a53e-38e14cec0200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:20.719285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfe7f7-7a5b-4677-a53e-38e14cec0200 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:20.719285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfe7f7-7a5b-4677-a53e-38e14cec0200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:20.720132 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bfe7f7-7a5b-4677-a53e-38e14cec0200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:20.726640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:20.731184 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bfe7f7-7a5b-4677-a53e-38e14cec0200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:20.732990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bfe7f7-7a5b-4677-a53e-38e14cec0200 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:20.754089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1134/2207] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:21.068191 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157204f-f5c7-40dc-85a1-934b787b9f68 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:21.070969 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9157204f-f5c7-40dc-85a1-934b787b9f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:21.091974 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9157204f-f5c7-40dc-85a1-934b787b9f68 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:21.097906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:21.099522 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:21.099984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:21.100847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d admin admin] PATH_INFO: `/v3/projects/a320d40801364c9f9843446a627dd421` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:21.102481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a320d40801364c9f9843446a627dd421)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:21.105858 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:21.174148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a320d40801364c9f9843446a627dd421', 'request_id': 'req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:21.183637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a320d40801364c9f9843446a627dd421', 'request_id': 'req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:21.195287 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a320d40801364c9f9843446a627dd421', 'request_id': 'req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:21.207094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d admin admin] The token cache is being invalidate because project a320d40801364c9f9843446a627dd421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:21.207326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:21.207962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3b42aeb9-8790-4b82-97c2-8bdeff94ad9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:21.209882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1074/2208] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:21 2022] DELETE /identity/v3/projects/a320d40801364c9f9843446a627dd421 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:21.220415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827c1fa-e944-4dc0-9ff7-5448eda368cc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:21.221157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827c1fa-e944-4dc0-9ff7-5448eda368cc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:21.221284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827c1fa-e944-4dc0-9ff7-5448eda368cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:21.255905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9827c1fa-e944-4dc0-9ff7-5448eda368cc None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:21.257246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9827c1fa-e944-4dc0-9ff7-5448eda368cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:21.327710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1135/2209] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:21.335728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2bf94-5421-4084-a6ca-eafa7c7ce79b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:21.336000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2bf94-5421-4084-a6ca-eafa7c7ce79b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:21.336298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2bf94-5421-4084-a6ca-eafa7c7ce79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:21.363978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-33f2bf94-5421-4084-a6ca-eafa7c7ce79b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:21.365626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-33f2bf94-5421-4084-a6ca-eafa7c7ce79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:21.383363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1075/2210] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:21.386378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827c1fa-e944-4dc0-9ff7-5448eda368cc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:21.399817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827c1fa-e944-4dc0-9ff7-5448eda368cc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:21.403841 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be84f2-2f6a-492f-b52f-1df503be1146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:21.404771 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be84f2-2f6a-492f-b52f-1df503be1146 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:21.405156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be84f2-2f6a-492f-b52f-1df503be1146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:21.405527 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be84f2-2f6a-492f-b52f-1df503be1146 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:21.406432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be84f2-2f6a-492f-b52f-1df503be1146 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:21.406965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be84f2-2f6a-492f-b52f-1df503be1146 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:21.408936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81be84f2-2f6a-492f-b52f-1df503be1146 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:21.436051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1136/2211] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:57:21.441560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2bf94-5421-4084-a6ca-eafa7c7ce79b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:21.456840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2bf94-5421-4084-a6ca-eafa7c7ce79b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:21.485235 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7182b-bf15-4bcf-9626-bafee1e8b7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:21.486602 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7182b-bf15-4bcf-9626-bafee1e8b7da admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:21.488865 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7182b-bf15-4bcf-9626-bafee1e8b7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:21.489378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d7182b-bf15-4bcf-9626-bafee1e8b7da admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:21.490771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7182b-bf15-4bcf-9626-bafee1e8b7da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:21.494525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d7182b-bf15-4bcf-9626-bafee1e8b7da admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:21.548913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1076/2212] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:21 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:21.557848 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be84f2-2f6a-492f-b52f-1df503be1146 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:21.582645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81be84f2-2f6a-492f-b52f-1df503be1146 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:21.712249 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beaafb-57f1-46a3-a530-a9565f7627c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:21.713378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beaafb-57f1-46a3-a530-a9565f7627c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:21.713747 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beaafb-57f1-46a3-a530-a9565f7627c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:21.715127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beaafb-57f1-46a3-a530-a9565f7627c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:21.716023 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0beaafb-57f1-46a3-a530-a9565f7627c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:21.718228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0beaafb-57f1-46a3-a530-a9565f7627c6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:21.751583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1137/2213] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:21 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:21.756815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7182b-bf15-4bcf-9626-bafee1e8b7da admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:21.772922 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d7182b-bf15-4bcf-9626-bafee1e8b7da admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:21.796631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7b2f1-56b0-4aef-bd42-68db981a9221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:21.798204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7b2f1-56b0-4aef-bd42-68db981a9221 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:21.798857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7b2f1-56b0-4aef-bd42-68db981a9221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:21.799131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d7b2f1-56b0-4aef-bd42-68db981a9221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:21.801166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7b2f1-56b0-4aef-bd42-68db981a9221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:21.803788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d7b2f1-56b0-4aef-bd42-68db981a9221 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:21.832137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1077/2214] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:21.832907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beaafb-57f1-46a3-a530-a9565f7627c6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:21.852724 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beaafb-57f1-46a3-a530-a9565f7627c6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:21.858542 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18c05e-78dd-4807-9f25-996299ee696f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:21.859222 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18c05e-78dd-4807-9f25-996299ee696f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:21.859517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18c05e-78dd-4807-9f25-996299ee696f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:21.860219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18c05e-78dd-4807-9f25-996299ee696f admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:21.860539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18c05e-78dd-4807-9f25-996299ee696f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:21.862202 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18c05e-78dd-4807-9f25-996299ee696f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:21.881610 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1138/2215] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:57:21.886982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7b2f1-56b0-4aef-bd42-68db981a9221 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:21.900653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d7b2f1-56b0-4aef-bd42-68db981a9221 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:21.906482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174de78-fee7-4beb-8b9e-d6f389ec5070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:21.907573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174de78-fee7-4beb-8b9e-d6f389ec5070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:21.908047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174de78-fee7-4beb-8b9e-d6f389ec5070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:21.908569 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174de78-fee7-4beb-8b9e-d6f389ec5070 admin admin] PATH_INFO: `/v3/projects/45aa73614bc447bdb1bfda38ad541ab3/users/2658206b0a934bd8861a49701ab41659/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:21.929313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174de78-fee7-4beb-8b9e-d6f389ec5070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45aa73614bc447bdb1bfda38ad541ab3, user_id=2658206b0a934bd8861a49701ab41659, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:21.932589 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174de78-fee7-4beb-8b9e-d6f389ec5070 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:21.936613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:21.956850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1078/2216] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:21 2022] PUT /identity/v3/projects/45aa73614bc447bdb1bfda38ad541ab3/users/2658206b0a934bd8861a49701ab41659/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:21.963249 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8452aa-b346-4438-887c-dbaed9e5148a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:21.963782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8452aa-b346-4438-887c-dbaed9e5148a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:21.964243 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8452aa-b346-4438-887c-dbaed9e5148a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:21.994887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4e8452aa-b346-4438-887c-dbaed9e5148a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:21.997158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4e8452aa-b346-4438-887c-dbaed9e5148a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:22.104335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:22.130133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1139/2217] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:22.142321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174de78-fee7-4beb-8b9e-d6f389ec5070 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:22.142901 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0174de78-fee7-4beb-8b9e-d6f389ec5070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:22.225091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174de78-fee7-4beb-8b9e-d6f389ec5070 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:22.229242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b2ea2-39ad-4373-9020-74c1cfee4466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:22.230131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b2ea2-39ad-4373-9020-74c1cfee4466 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:22.230415 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b2ea2-39ad-4373-9020-74c1cfee4466 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:22.230693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b2ea2-39ad-4373-9020-74c1cfee4466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:22.241859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b2ea2-39ad-4373-9020-74c1cfee4466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:22.243329 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1b2ea2-39ad-4373-9020-74c1cfee4466 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:22.256069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1079/2218] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:25.424705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d84ce-b0fb-4cef-be05-59f9e73cc440 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:25.425371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d84ce-b0fb-4cef-be05-59f9e73cc440 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:25.426115 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d84ce-b0fb-4cef-be05-59f9e73cc440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:25.458782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c07d84ce-b0fb-4cef-be05-59f9e73cc440 None None] MFA Rules not processed for user `53f1188f2446493baae7dc79bbb66ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:25.460821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c07d84ce-b0fb-4cef-be05-59f9e73cc440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:25.574588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1140/2219] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:27.121977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a062f-f648-4f77-a0cb-3e9747703529 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:27.122501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a062f-f648-4f77-a0cb-3e9747703529 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:27.123086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a062f-f648-4f77-a0cb-3e9747703529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:27.205448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3a0a062f-f648-4f77-a0cb-3e9747703529 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:27.206803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3a0a062f-f648-4f77-a0cb-3e9747703529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:27.382050 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1080/2220] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:57:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:27.488719 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d84ce-b0fb-4cef-be05-59f9e73cc440 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:27.490987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c07d84ce-b0fb-4cef-be05-59f9e73cc440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:27.684256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d84ce-b0fb-4cef-be05-59f9e73cc440 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:27.696915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85223f-1434-4a40-9804-03614f26557e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:27.698180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85223f-1434-4a40-9804-03614f26557e service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:27.698851 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85223f-1434-4a40-9804-03614f26557e service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:27.699551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c85223f-1434-4a40-9804-03614f26557e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:27.732973 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85223f-1434-4a40-9804-03614f26557e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:27.734725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c85223f-1434-4a40-9804-03614f26557e service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:27.760588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1141/2221] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:57:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:57:29.280408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef483c-bc49-4059-9333-3fa3661aa965 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.281437 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef483c-bc49-4059-9333-3fa3661aa965 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.281729 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ef483c-bc49-4059-9333-3fa3661aa965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.318643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-30ef483c-bc49-4059-9333-3fa3661aa965 None None] MFA Rules not processed for user `2658206b0a934bd8861a49701ab41659`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:29.320107 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-30ef483c-bc49-4059-9333-3fa3661aa965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:29.428814 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1081/2222] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:29.432062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85223f-1434-4a40-9804-03614f26557e service swift] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:29.444772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c85223f-1434-4a40-9804-03614f26557e service swift] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:29.454939 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49192a39-5e7e-42a4-9458-e9bdb707c393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:29.456897 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49192a39-5e7e-42a4-9458-e9bdb707c393 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.458455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49192a39-5e7e-42a4-9458-e9bdb707c393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.458766 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49192a39-5e7e-42a4-9458-e9bdb707c393 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.461647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49192a39-5e7e-42a4-9458-e9bdb707c393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:29.465275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49192a39-5e7e-42a4-9458-e9bdb707c393 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:29.541587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1142/2223] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:29 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:29.542441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef483c-bc49-4059-9333-3fa3661aa965 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:29.561265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ef483c-bc49-4059-9333-3fa3661aa965 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:29.589099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:29.754557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26659b4f-d481-4408-8ae3-229a359de48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:29.755254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26659b4f-d481-4408-8ae3-229a359de48a admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.755548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26659b4f-d481-4408-8ae3-229a359de48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.755947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26659b4f-d481-4408-8ae3-229a359de48a admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.756534 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26659b4f-d481-4408-8ae3-229a359de48a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:29.759708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26659b4f-d481-4408-8ae3-229a359de48a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:29.827050 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1082/2224] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:29 2022] POST /identity/v3/users => generated 298 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:29.831282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49192a39-5e7e-42a4-9458-e9bdb707c393 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:29.858816 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49192a39-5e7e-42a4-9458-e9bdb707c393 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:29.865557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f10a8f-ba15-4c0a-98a0-14d19f86f180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:29.866761 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f10a8f-ba15-4c0a-98a0-14d19f86f180 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.867214 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f10a8f-ba15-4c0a-98a0-14d19f86f180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.867649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f10a8f-ba15-4c0a-98a0-14d19f86f180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.869044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f10a8f-ba15-4c0a-98a0-14d19f86f180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:29.871479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f10a8f-ba15-4c0a-98a0-14d19f86f180 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:29.884717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1143/2225] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:29.888244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26659b4f-d481-4408-8ae3-229a359de48a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:29.905028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26659b4f-d481-4408-8ae3-229a359de48a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:29.911250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bd80b-6801-4ee6-947a-7bf2c4794e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:29.912574 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bd80b-6801-4ee6-947a-7bf2c4794e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:29.912895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bd80b-6801-4ee6-947a-7bf2c4794e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:29.913180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9bd80b-6801-4ee6-947a-7bf2c4794e61 admin admin] PATH_INFO: `/v3/projects/1c4334d0e4a245f0a0a1ab8d6e35a89a/users/d990402118ea4227aa0f8feda9e42e61/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:29.932304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bd80b-6801-4ee6-947a-7bf2c4794e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4334d0e4a245f0a0a1ab8d6e35a89a, user_id=d990402118ea4227aa0f8feda9e42e61, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:29.935661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9bd80b-6801-4ee6-947a-7bf2c4794e61 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:29.960069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1083/2226] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:29 2022] PUT /identity/v3/projects/1c4334d0e4a245f0a0a1ab8d6e35a89a/users/d990402118ea4227aa0f8feda9e42e61/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:29.966112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f10a8f-ba15-4c0a-98a0-14d19f86f180 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:29.987340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f10a8f-ba15-4c0a-98a0-14d19f86f180 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:30.093923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:30.121487 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae5ed7-1667-4815-884e-7997bd695b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:30.122560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae5ed7-1667-4815-884e-7997bd695b3b admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.123003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae5ed7-1667-4815-884e-7997bd695b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.123356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae5ed7-1667-4815-884e-7997bd695b3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.124302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae5ed7-1667-4815-884e-7997bd695b3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:30.126575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae5ed7-1667-4815-884e-7997bd695b3b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:30.143925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1144/2227] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:30.148976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bd80b-6801-4ee6-947a-7bf2c4794e61 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:30.165940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9bd80b-6801-4ee6-947a-7bf2c4794e61 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:30.174795 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c9ca4-ae68-43d2-9dbc-27fffdae48ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:30.175769 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c9ca4-ae68-43d2-9dbc-27fffdae48ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.177809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c9ca4-ae68-43d2-9dbc-27fffdae48ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.177809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c9ca4-ae68-43d2-9dbc-27fffdae48ab admin admin] PATH_INFO: `/v3/projects/1c4334d0e4a245f0a0a1ab8d6e35a89a/users/d990402118ea4227aa0f8feda9e42e61/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.180375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c9ca4-ae68-43d2-9dbc-27fffdae48ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4334d0e4a245f0a0a1ab8d6e35a89a, user_id=d990402118ea4227aa0f8feda9e42e61, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:30.184739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c9ca4-ae68-43d2-9dbc-27fffdae48ab admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:30.204512 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1084/2228] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:30 2022] PUT /identity/v3/projects/1c4334d0e4a245f0a0a1ab8d6e35a89a/users/d990402118ea4227aa0f8feda9e42e61/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:30.806648 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd11d25-46a4-4f1a-9a32-9753772be198 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:30.807246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd11d25-46a4-4f1a-9a32-9753772be198 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:30.807343 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd11d25-46a4-4f1a-9a32-9753772be198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:30.853432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-6cd11d25-46a4-4f1a-9a32-9753772be198 None None] MFA Rules not processed for user `f7cf8ed5de4b41b19b6f25299087e1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:30.854761 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6cd11d25-46a4-4f1a-9a32-9753772be198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:31.028401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1145/2229] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:32.276257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c9ca4-ae68-43d2-9dbc-27fffdae48ab admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:32.280009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2b5c9ca4-ae68-43d2-9dbc-27fffdae48ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:32.618331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c9ca4-ae68-43d2-9dbc-27fffdae48ab admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:32.634539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:32.635520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:32.636075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:32.636231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] PATH_INFO: `/v3/users/158ea46d74bc4bcf95f81e0c6c878e88` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:32.644872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] RBAC: Authorizing `identity:delete_user(user_id=158ea46d74bc4bcf95f81e0c6c878e88)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:32.657891 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:32.806561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '158ea46d74bc4bcf95f81e0c6c878e88'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.808320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '158ea46d74bc4bcf95f81e0c6c878e88'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.812654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '158ea46d74bc4bcf95f81e0c6c878e88'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.862108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '158ea46d74bc4bcf95f81e0c6c878e88'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.928389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '158ea46d74bc4bcf95f81e0c6c878e88'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:32.942656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:32.996298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1085/2230] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:57:32 2022] DELETE /identity/v3/users/158ea46d74bc4bcf95f81e0c6c878e88 => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:33.001337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07a1d4-1a73-4415-8ffd-029f06efc1ae None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.002250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07a1d4-1a73-4415-8ffd-029f06efc1ae None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.002708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f07a1d4-1a73-4415-8ffd-029f06efc1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.074111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1f07a1d4-1a73-4415-8ffd-029f06efc1ae None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:33.086507 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1f07a1d4-1a73-4415-8ffd-029f06efc1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:33.283280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1146/2231] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:33.354397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:33.356763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:33.504842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea792cc-3255-457f-8c08-0768c7c7b51b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:33.513832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd538b-a817-4e7b-898b-0e2707aaab9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:33.514494 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd538b-a817-4e7b-898b-0e2707aaab9b service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:33.514778 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd538b-a817-4e7b-898b-0e2707aaab9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:33.515099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd538b-a817-4e7b-898b-0e2707aaab9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:33.542144 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd538b-a817-4e7b-898b-0e2707aaab9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:33.543657 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd538b-a817-4e7b-898b-0e2707aaab9b service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:33.578432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1086/2232] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:36.587624 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07a1d4-1a73-4415-8ffd-029f06efc1ae None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:36.590530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1f07a1d4-1a73-4415-8ffd-029f06efc1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:36.628751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f07a1d4-1a73-4415-8ffd-029f06efc1ae None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:36.705305 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:36.723379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:36.724036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:36.725162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322 admin admin] PATH_INFO: `/v3/projects/5dac2d4d38da494a8cd4cd136fe54f5b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:36.734888 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dac2d4d38da494a8cd4cd136fe54f5b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:36.740389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:36.883771 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dac2d4d38da494a8cd4cd136fe54f5b', 'request_id': 'req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:36.905116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dac2d4d38da494a8cd4cd136fe54f5b', 'request_id': 'req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:36.914195 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dac2d4d38da494a8cd4cd136fe54f5b', 'request_id': 'req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:36.949073 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322 admin admin] The token cache is being invalidate because project 5dac2d4d38da494a8cd4cd136fe54f5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:57:36.949567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:36.950800 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-cff606b2-ba0d-4ce3-97f4-1ad58eda4322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:57:36.964787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1147/2233] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:57:36 2022] DELETE /identity/v3/projects/5dac2d4d38da494a8cd4cd136fe54f5b => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:36.968340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ad409-3e81-4689-a33a-4c0cd36147eb None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:36.968993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ad409-3e81-4689-a33a-4c0cd36147eb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:36.969380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ad409-3e81-4689-a33a-4c0cd36147eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.003835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-899ad409-3e81-4689-a33a-4c0cd36147eb None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:37.005811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-899ad409-3e81-4689-a33a-4c0cd36147eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:37.143998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1087/2234] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:37.150599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd0254-3021-47b1-bc58-6daac98c290b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.151413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd0254-3021-47b1-bc58-6daac98c290b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.151867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd0254-3021-47b1-bc58-6daac98c290b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.255289 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0bfd0254-3021-47b1-bc58-6daac98c290b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:37.256776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0bfd0254-3021-47b1-bc58-6daac98c290b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:37.299551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1148/2235] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:37.304722 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ad409-3e81-4689-a33a-4c0cd36147eb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.323882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ad409-3e81-4689-a33a-4c0cd36147eb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:37.335320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822a257-1cfc-4eeb-a491-cb14a3638144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.336859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822a257-1cfc-4eeb-a491-cb14a3638144 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.337149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822a257-1cfc-4eeb-a491-cb14a3638144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.337432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822a257-1cfc-4eeb-a491-cb14a3638144 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.340575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7822a257-1cfc-4eeb-a491-cb14a3638144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:57:37.342736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7822a257-1cfc-4eeb-a491-cb14a3638144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:37.344403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7822a257-1cfc-4eeb-a491-cb14a3638144 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:37.381126 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1088/2236] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:57:37.383856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd0254-3021-47b1-bc58-6daac98c290b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.398881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd0254-3021-47b1-bc58-6daac98c290b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:37.411411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8d468-e966-49f1-bdf1-d8bcc5dc0977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.412203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8d468-e966-49f1-bdf1-d8bcc5dc0977 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.412569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8d468-e966-49f1-bdf1-d8bcc5dc0977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.412935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8d468-e966-49f1-bdf1-d8bcc5dc0977 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.413735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8d468-e966-49f1-bdf1-d8bcc5dc0977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:37.416977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8d468-e966-49f1-bdf1-d8bcc5dc0977 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:37.441827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:37.481151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1149/2237] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:57:37 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 11:57:37.486066 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822a257-1cfc-4eeb-a491-cb14a3638144 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.502945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822a257-1cfc-4eeb-a491-cb14a3638144 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:37.641234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24112b5-701d-4d81-a03f-70304caddc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.642253 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24112b5-701d-4d81-a03f-70304caddc45 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.642771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24112b5-701d-4d81-a03f-70304caddc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.643396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24112b5-701d-4d81-a03f-70304caddc45 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.644655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24112b5-701d-4d81-a03f-70304caddc45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:37.647021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24112b5-701d-4d81-a03f-70304caddc45 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:37.736145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1089/2238] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:57:37 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:57:37.740662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8d468-e966-49f1-bdf1-d8bcc5dc0977 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.767316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8d468-e966-49f1-bdf1-d8bcc5dc0977 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:37.782172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d4d38-87fb-4e6c-aaef-feae62256932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.786465 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d4d38-87fb-4e6c-aaef-feae62256932 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.787001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d4d38-87fb-4e6c-aaef-feae62256932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.787398 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d4d38-87fb-4e6c-aaef-feae62256932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.791514 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d4d38-87fb-4e6c-aaef-feae62256932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:37.793747 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d4d38-87fb-4e6c-aaef-feae62256932 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:37.841382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1150/2239] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:37.843689 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24112b5-701d-4d81-a03f-70304caddc45 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.890306 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24112b5-701d-4d81-a03f-70304caddc45 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:37.920973 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9975f02c-9fa3-4ca0-8b7e-dee578baa038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.926764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975f02c-9fa3-4ca0-8b7e-dee578baa038 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:37.931549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975f02c-9fa3-4ca0-8b7e-dee578baa038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:37.934835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9975f02c-9fa3-4ca0-8b7e-dee578baa038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:37.936223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975f02c-9fa3-4ca0-8b7e-dee578baa038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:37.940127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9975f02c-9fa3-4ca0-8b7e-dee578baa038 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:37.965493 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1090/2240] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:57:37 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:57:37.969580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d4d38-87fb-4e6c-aaef-feae62256932 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:37.987559 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d4d38-87fb-4e6c-aaef-feae62256932 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:37.997989 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48499250-7510-4ac1-997d-be748731c3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:37.998920 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48499250-7510-4ac1-997d-be748731c3ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:38.002591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48499250-7510-4ac1-997d-be748731c3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:38.004357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48499250-7510-4ac1-997d-be748731c3ac admin admin] PATH_INFO: `/v3/projects/d410564d1ba94f7883cd07fcf16be765/users/74464ea11cbc4d44999583f93283a411/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:38.049258 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48499250-7510-4ac1-997d-be748731c3ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d410564d1ba94f7883cd07fcf16be765, user_id=74464ea11cbc4d44999583f93283a411, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:38.052216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48499250-7510-4ac1-997d-be748731c3ac admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:38.085121 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1151/2241] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:57:37 2022] PUT /identity/v3/projects/d410564d1ba94f7883cd07fcf16be765/users/74464ea11cbc4d44999583f93283a411/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:57:38.093097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bc4c2-a07c-4a54-af4f-fc8c329d1096 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:38.093661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bc4c2-a07c-4a54-af4f-fc8c329d1096 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:38.093967 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bc4c2-a07c-4a54-af4f-fc8c329d1096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:38.123323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:38.201390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-fc6bc4c2-a07c-4a54-af4f-fc8c329d1096 None None] MFA Rules not processed for user `74464ea11cbc4d44999583f93283a411`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:38.203040 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fc6bc4c2-a07c-4a54-af4f-fc8c329d1096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:38.353359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1091/2242] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:38.361941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48499250-7510-4ac1-997d-be748731c3ac admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:38.364306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-48499250-7510-4ac1-997d-be748731c3ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:38.540073 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48499250-7510-4ac1-997d-be748731c3ac admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:38.552899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee68bca-1000-437d-b202-47b1d615f113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:38.556450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee68bca-1000-437d-b202-47b1d615f113 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:38.556450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee68bca-1000-437d-b202-47b1d615f113 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:38.556450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee68bca-1000-437d-b202-47b1d615f113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:38.580142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee68bca-1000-437d-b202-47b1d615f113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:38.581668 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee68bca-1000-437d-b202-47b1d615f113 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:38.626606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1152/2243] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:40.653928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfe590-bee4-4b0b-a704-7d0ac39a7166 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:40.654501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfe590-bee4-4b0b-a704-7d0ac39a7166 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:40.654501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfe590-bee4-4b0b-a704-7d0ac39a7166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:40.687929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9fdfe590-bee4-4b0b-a704-7d0ac39a7166 None None] MFA Rules not processed for user `74464ea11cbc4d44999583f93283a411`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:40.689827 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9fdfe590-bee4-4b0b-a704-7d0ac39a7166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:40.710949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1092/2244] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:44.279594 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee68bca-1000-437d-b202-47b1d615f113 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:44.281170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-aee68bca-1000-437d-b202-47b1d615f113 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:44.456710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f2fb6-59e6-41a9-8bd5-a0f7e5c2fee1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:44.456710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f2fb6-59e6-41a9-8bd5-a0f7e5c2fee1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:44.456710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f2fb6-59e6-41a9-8bd5-a0f7e5c2fee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:44.522793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1b3f2fb6-59e6-41a9-8bd5-a0f7e5c2fee1 None None] MFA Rules not processed for user `d990402118ea4227aa0f8feda9e42e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:44.524058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1b3f2fb6-59e6-41a9-8bd5-a0f7e5c2fee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:44.527388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee68bca-1000-437d-b202-47b1d615f113 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:44.540795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533b2e7-3f8c-4030-a1e8-ac19182e8852 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:44.542044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533b2e7-3f8c-4030-a1e8-ac19182e8852 service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:44.542335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533b2e7-3f8c-4030-a1e8-ac19182e8852 service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:44.542616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533b2e7-3f8c-4030-a1e8-ac19182e8852 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:44.557985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7533b2e7-3f8c-4030-a1e8-ac19182e8852 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:44.585687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533b2e7-3f8c-4030-a1e8-ac19182e8852 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:44.587389 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7533b2e7-3f8c-4030-a1e8-ac19182e8852 service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:44.611260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1153/2245] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:44.789923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1093/2246] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:44.800439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533b2e7-3f8c-4030-a1e8-ac19182e8852 service placement] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:44.801950 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7533b2e7-3f8c-4030-a1e8-ac19182e8852 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:44.888543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:44.965966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533b2e7-3f8c-4030-a1e8-ac19182e8852 service placement] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:44.970528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6a7f1-4886-4d5e-bf57-ac2f8c5a6b26 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74fdc12d728d45018ea668a4073da3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:44.971536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6a7f1-4886-4d5e-bf57-ac2f8c5a6b26 service glance] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:44.972437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6a7f1-4886-4d5e-bf57-ac2f8c5a6b26 service glance] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:44.973392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6a7f1-4886-4d5e-bf57-ac2f8c5a6b26 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:44.996362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1bf6a7f1-4886-4d5e-bf57-ac2f8c5a6b26 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:45.168056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6a7f1-4886-4d5e-bf57-ac2f8c5a6b26 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:45.170452 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6a7f1-4886-4d5e-bf57-ac2f8c5a6b26 service glance] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:45.208391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1154/2247] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:57:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:57:45.411108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476fe1b-f5df-46b7-a0a0-67ca89f1dcd2 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:45.411480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476fe1b-f5df-46b7-a0a0-67ca89f1dcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:45.411848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476fe1b-f5df-46b7-a0a0-67ca89f1dcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:45.450549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-4476fe1b-f5df-46b7-a0a0-67ca89f1dcd2 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:45.451853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4476fe1b-f5df-46b7-a0a0-67ca89f1dcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:45.537805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1094/2248] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:57:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:45.577617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6a7f1-4886-4d5e-bf57-ac2f8c5a6b26 service glance] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:57:45.579486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1bf6a7f1-4886-4d5e-bf57-ac2f8c5a6b26 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:45.709078 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6a7f1-4886-4d5e-bf57-ac2f8c5a6b26 service glance] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:57:45.715499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca8a75-0ae7-4da8-8156-acd6c3dc3d55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:57:45.716483 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8a75-0ae7-4da8-8156-acd6c3dc3d55 service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:45.717003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8a75-0ae7-4da8-8156-acd6c3dc3d55 service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:45.717493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8a75-0ae7-4da8-8156-acd6c3dc3d55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:45.730190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca8a75-0ae7-4da8-8156-acd6c3dc3d55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:57:45.732320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca8a75-0ae7-4da8-8156-acd6c3dc3d55 service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:57:45.747018 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1155/2249] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:57:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:57:45.936560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852221b-b5b7-4f3d-a505-cdf6887a81b7 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:57:45.936955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852221b-b5b7-4f3d-a505-cdf6887a81b7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:57:45.937314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4852221b-b5b7-4f3d-a505-cdf6887a81b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:57:45.980044 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-4852221b-b5b7-4f3d-a505-cdf6887a81b7 None None] MFA Rules not processed for user `9cc891cc8172490394d2bfa3c8d1dcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:57:45.981775 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4852221b-b5b7-4f3d-a505-cdf6887a81b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:57:45.994671 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:46.127761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1095/2250] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:57:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:57:55.739521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=68 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:57:56.118140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=20 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:02.335885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba1124-53a3-481e-bc26-c805a2a52a6b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:02.336449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba1124-53a3-481e-bc26-c805a2a52a6b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:02.336449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ba1124-53a3-481e-bc26-c805a2a52a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:02.423998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-00ba1124-53a3-481e-bc26-c805a2a52a6b None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:02.425618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-00ba1124-53a3-481e-bc26-c805a2a52a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:02.452979 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1156/2251] 10.222.0.67 () {58 vars in 967 bytes} [Thu Jun 23 11:58:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:02.466156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852221b-b5b7-4f3d-a505-cdf6887a81b7 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:02.491482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4852221b-b5b7-4f3d-a505-cdf6887a81b7 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:02.497341 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f098c7-7a38-4aa3-ae89-b608f8db1a8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:02.498389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f098c7-7a38-4aa3-ae89-b608f8db1a8b service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:02.498873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f098c7-7a38-4aa3-ae89-b608f8db1a8b service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:02.499483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f098c7-7a38-4aa3-ae89-b608f8db1a8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:02.518575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f098c7-7a38-4aa3-ae89-b608f8db1a8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:02.522072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f098c7-7a38-4aa3-ae89-b608f8db1a8b service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:02.548323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1096/2252] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:58:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:58:02.596082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24956f2-92f3-42b9-b150-b0951b218919 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:02.596602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24956f2-92f3-42b9-b150-b0951b218919 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:02.598553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24956f2-92f3-42b9-b150-b0951b218919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:02.621552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a24956f2-92f3-42b9-b150-b0951b218919 None None] MFA Rules not processed for user `2658206b0a934bd8861a49701ab41659`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:02.623261 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a24956f2-92f3-42b9-b150-b0951b218919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:02.643239 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1157/2253] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:58:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:02.690518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b0e2b-133d-4ccf-8349-b5e3922ce813 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:02.691193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b0e2b-133d-4ccf-8349-b5e3922ce813 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:02.691599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b0e2b-133d-4ccf-8349-b5e3922ce813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:02.716130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-291b0e2b-133d-4ccf-8349-b5e3922ce813 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:02.718662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-291b0e2b-133d-4ccf-8349-b5e3922ce813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:02.727867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1097/2254] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:58:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:02.754976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24956f2-92f3-42b9-b150-b0951b218919 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:02.769378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24956f2-92f3-42b9-b150-b0951b218919 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:02.774872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911ae72-da35-4e1b-b1f8-511b776b0d84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:02.775548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911ae72-da35-4e1b-b1f8-511b776b0d84 service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:02.775833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911ae72-da35-4e1b-b1f8-511b776b0d84 service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:02.776111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911ae72-da35-4e1b-b1f8-511b776b0d84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:02.788171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911ae72-da35-4e1b-b1f8-511b776b0d84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:02.789882 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911ae72-da35-4e1b-b1f8-511b776b0d84 service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:02.799570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1158/2255] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:58:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:58:05.661014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b0e2b-133d-4ccf-8349-b5e3922ce813 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:05.663162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-291b0e2b-133d-4ccf-8349-b5e3922ce813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:05.853299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b0e2b-133d-4ccf-8349-b5e3922ce813 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:05.869503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:05.871198 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:05.875072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:05.877401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] PATH_INFO: `/v3/users/23a9ad25030045b9b5dad89eadca1b9c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:05.881034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a9ad25030045b9b5dad89eadca1b9c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:05.894752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:06.060716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a9ad25030045b9b5dad89eadca1b9c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.061627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a9ad25030045b9b5dad89eadca1b9c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.062357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a9ad25030045b9b5dad89eadca1b9c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.098946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a9ad25030045b9b5dad89eadca1b9c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.128147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a9ad25030045b9b5dad89eadca1b9c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.156254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1098/2256] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:58:05 2022] DELETE /identity/v3/users/23a9ad25030045b9b5dad89eadca1b9c => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:58:06.264531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414eef83-9bce-414b-b0be-a96b61594d11 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:06.265060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414eef83-9bce-414b-b0be-a96b61594d11 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:06.265334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414eef83-9bce-414b-b0be-a96b61594d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:06.298159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-414eef83-9bce-414b-b0be-a96b61594d11 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:06.299832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-414eef83-9bce-414b-b0be-a96b61594d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:06.404094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1159/2257] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:58:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:06.425404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:06.427019 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:06.441394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414eef83-9bce-414b-b0be-a96b61594d11 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:06.444138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-414eef83-9bce-414b-b0be-a96b61594d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:06.594211 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:06.606521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414eef83-9bce-414b-b0be-a96b61594d11 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:06.617805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d246d1c-6443-499e-8788-22870d7fb1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:06.619490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d246d1c-6443-499e-8788-22870d7fb1e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:06.620526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d246d1c-6443-499e-8788-22870d7fb1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:06.621088 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d246d1c-6443-499e-8788-22870d7fb1e9 admin admin] PATH_INFO: `/v3/projects/d4f7063c9ea64b569c3699a45ad0bb2e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:06.624654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d246d1c-6443-499e-8788-22870d7fb1e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4f7063c9ea64b569c3699a45ad0bb2e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:06.626994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d246d1c-6443-499e-8788-22870d7fb1e9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:06.640035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1697e60c-f389-4231-9e11-a5d897545324 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:06.648322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec5ca74-c811-44ca-bc1e-626b5c800022 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:06.649028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5ca74-c811-44ca-bc1e-626b5c800022 service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:06.649816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5ca74-c811-44ca-bc1e-626b5c800022 service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:06.650641 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec5ca74-c811-44ca-bc1e-626b5c800022 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:06.683221 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5ca74-c811-44ca-bc1e-626b5c800022 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:06.684878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec5ca74-c811-44ca-bc1e-626b5c800022 service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:06.709819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1099/2258] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:58:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:58:06.793370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:06.796687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6d246d1c-6443-499e-8788-22870d7fb1e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4f7063c9ea64b569c3699a45ad0bb2e', 'request_id': 'req-6d246d1c-6443-499e-8788-22870d7fb1e9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.811108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6d246d1c-6443-499e-8788-22870d7fb1e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4f7063c9ea64b569c3699a45ad0bb2e', 'request_id': 'req-6d246d1c-6443-499e-8788-22870d7fb1e9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.835302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6d246d1c-6443-499e-8788-22870d7fb1e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4f7063c9ea64b569c3699a45ad0bb2e', 'request_id': 'req-6d246d1c-6443-499e-8788-22870d7fb1e9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.871208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6d246d1c-6443-499e-8788-22870d7fb1e9 admin admin] The token cache is being invalidate because project d4f7063c9ea64b569c3699a45ad0bb2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:58:06.879723 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6d246d1c-6443-499e-8788-22870d7fb1e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.882619 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6d246d1c-6443-499e-8788-22870d7fb1e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:58:06.889013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1160/2259] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:58:06 2022] DELETE /identity/v3/projects/d4f7063c9ea64b569c3699a45ad0bb2e => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:58:06.890785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b75d2d-297d-46e0-9980-0503af195eeb None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:06.891088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b75d2d-297d-46e0-9980-0503af195eeb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:06.891376 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b75d2d-297d-46e0-9980-0503af195eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:06.924526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-97b75d2d-297d-46e0-9980-0503af195eeb None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:06.925864 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-97b75d2d-297d-46e0-9980-0503af195eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:07.222812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1100/2260] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:58:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:07.230463 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2624c37-2ae4-4e0c-b5bd-60dcc5e73941 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:07.230821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2624c37-2ae4-4e0c-b5bd-60dcc5e73941 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:07.231033 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2624c37-2ae4-4e0c-b5bd-60dcc5e73941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:07.270257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a2624c37-2ae4-4e0c-b5bd-60dcc5e73941 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:07.272765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a2624c37-2ae4-4e0c-b5bd-60dcc5e73941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:07.301258 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1161/2261] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:58:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:07.305244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b75d2d-297d-46e0-9980-0503af195eeb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:07.324311 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b75d2d-297d-46e0-9980-0503af195eeb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:07.345975 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a1439-8823-408e-9693-5e458acd80cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:07.347616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a1439-8823-408e-9693-5e458acd80cc admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:07.347910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a1439-8823-408e-9693-5e458acd80cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:07.348188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a1439-8823-408e-9693-5e458acd80cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:07.350167 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a1439-8823-408e-9693-5e458acd80cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:58:07.354576 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a1439-8823-408e-9693-5e458acd80cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:07.380028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a1439-8823-408e-9693-5e458acd80cc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:07.437168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1101/2262] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:58:07.438339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2624c37-2ae4-4e0c-b5bd-60dcc5e73941 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:07.449563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2624c37-2ae4-4e0c-b5bd-60dcc5e73941 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:07.483441 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29408b0e-aacd-4af5-a6b7-96501a80c132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:07.484134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29408b0e-aacd-4af5-a6b7-96501a80c132 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:07.484431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29408b0e-aacd-4af5-a6b7-96501a80c132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:07.485134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29408b0e-aacd-4af5-a6b7-96501a80c132 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:07.486016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29408b0e-aacd-4af5-a6b7-96501a80c132 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:07.488412 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29408b0e-aacd-4af5-a6b7-96501a80c132 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:07.576306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1162/2263] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:58:07 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:58:07.580369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a1439-8823-408e-9693-5e458acd80cc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:07.598063 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a1439-8823-408e-9693-5e458acd80cc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:07.781587 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6584f-b823-4b8a-81b4-686752317dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:07.783632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6584f-b823-4b8a-81b4-686752317dff admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:07.787384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6584f-b823-4b8a-81b4-686752317dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:07.787999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6584f-b823-4b8a-81b4-686752317dff admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:07.789136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6584f-b823-4b8a-81b4-686752317dff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:07.791886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6584f-b823-4b8a-81b4-686752317dff admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:07.851930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1102/2264] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:58:07 2022] POST /identity/v3/users => generated 286 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:58:07.856576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29408b0e-aacd-4af5-a6b7-96501a80c132 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:07.868011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29408b0e-aacd-4af5-a6b7-96501a80c132 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:07.881256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bcc2f-c9ce-4470-9db1-2d62a0378115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:07.881914 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bcc2f-c9ce-4470-9db1-2d62a0378115 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:07.882212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bcc2f-c9ce-4470-9db1-2d62a0378115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:07.882497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bcc2f-c9ce-4470-9db1-2d62a0378115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:07.883249 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bcc2f-c9ce-4470-9db1-2d62a0378115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:07.907902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04bcc2f-c9ce-4470-9db1-2d62a0378115 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:07.944834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1163/2265] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:58:07.948589 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6584f-b823-4b8a-81b4-686752317dff admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:07.959455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6584f-b823-4b8a-81b4-686752317dff admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:07.965111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bb4ae-74f0-44fc-828c-bb08af0e09bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:07.965870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bb4ae-74f0-44fc-828c-bb08af0e09bf admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:07.966231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bb4ae-74f0-44fc-828c-bb08af0e09bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:07.966586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bb4ae-74f0-44fc-828c-bb08af0e09bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:07.968548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bb4ae-74f0-44fc-828c-bb08af0e09bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:07.970993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939bb4ae-74f0-44fc-828c-bb08af0e09bf admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:07.986238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1103/2266] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:58:07.987547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bcc2f-c9ce-4470-9db1-2d62a0378115 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:08.002405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04bcc2f-c9ce-4470-9db1-2d62a0378115 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:08.007409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2cb35-ca0d-4d56-8cf6-a9f38f0fc7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:08.008703 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2cb35-ca0d-4d56-8cf6-a9f38f0fc7b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:08.008703 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2cb35-ca0d-4d56-8cf6-a9f38f0fc7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:08.008940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2cb35-ca0d-4d56-8cf6-a9f38f0fc7b8 admin admin] PATH_INFO: `/v3/projects/e5e31b080963402f8f2553fbc16e4311/users/85138dda70fd4c4697289e5bfd137010/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:08.046742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2cb35-ca0d-4d56-8cf6-a9f38f0fc7b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e31b080963402f8f2553fbc16e4311, user_id=85138dda70fd4c4697289e5bfd137010, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:08.049979 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2cb35-ca0d-4d56-8cf6-a9f38f0fc7b8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:08.075764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1164/2267] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:58:07 2022] PUT /identity/v3/projects/e5e31b080963402f8f2553fbc16e4311/users/85138dda70fd4c4697289e5bfd137010/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:58:08.087383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351233af-c55a-48b2-9165-2fda9d9b3d67 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:08.092615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351233af-c55a-48b2-9165-2fda9d9b3d67 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:08.093317 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351233af-c55a-48b2-9165-2fda9d9b3d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:08.195462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-351233af-c55a-48b2-9165-2fda9d9b3d67 None None] MFA Rules not processed for user `85138dda70fd4c4697289e5bfd137010`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:08.197252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-351233af-c55a-48b2-9165-2fda9d9b3d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:08.331023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:08.477958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1104/2268] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:58:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:16.071536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2cb35-ca0d-4d56-8cf6-a9f38f0fc7b8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:16.073834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ddb2cb35-ca0d-4d56-8cf6-a9f38f0fc7b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:16.187214 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2cb35-ca0d-4d56-8cf6-a9f38f0fc7b8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:16.190855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead79c6-cf6b-4095-b65c-5390f4c13071 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74fdc12d728d45018ea668a4073da3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:16.191499 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead79c6-cf6b-4095-b65c-5390f4c13071 service glance] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:16.191801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead79c6-cf6b-4095-b65c-5390f4c13071 service glance] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:16.192065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead79c6-cf6b-4095-b65c-5390f4c13071 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:16.201407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead79c6-cf6b-4095-b65c-5390f4c13071 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:16.203093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead79c6-cf6b-4095-b65c-5390f4c13071 service glance] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:16.213234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1165/2269] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:58:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:58:18.468019 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=16 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:26.207980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=69,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:37.690009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f6324-83e7-4bfb-8051-645a2deb497f None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:37.690506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f6324-83e7-4bfb-8051-645a2deb497f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:37.690616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55f6324-83e7-4bfb-8051-645a2deb497f None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:37.692591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1105/2270] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 11:58:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:58:37.698843 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598506ed-e7f3-4a29-ace6-7e86076ec14a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:37.699943 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598506ed-e7f3-4a29-ace6-7e86076ec14a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:37.700444 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598506ed-e7f3-4a29-ace6-7e86076ec14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:37.762812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-598506ed-e7f3-4a29-ace6-7e86076ec14a None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:37.763993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-598506ed-e7f3-4a29-ace6-7e86076ec14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:37.894612 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1166/2271] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 11:58:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:58:37.908543 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f6324-83e7-4bfb-8051-645a2deb497f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:37.910565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a55f6324-83e7-4bfb-8051-645a2deb497f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:37.925310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55f6324-83e7-4bfb-8051-645a2deb497f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:37.929446 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b487ce-608b-45b4-8add-12ca671a6394 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74fdc12d728d45018ea668a4073da3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:37.930100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b487ce-608b-45b4-8add-12ca671a6394 service glance] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:37.930390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b487ce-608b-45b4-8add-12ca671a6394 service glance] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:37.930674 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b487ce-608b-45b4-8add-12ca671a6394 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:37.941976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b487ce-608b-45b4-8add-12ca671a6394 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:37.943434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b487ce-608b-45b4-8add-12ca671a6394 service glance] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:37.971268 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1106/2272] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:58:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:58:38.049016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ca5fb-1a83-42b3-b733-6dc686c9cfda None None] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:38.049317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ca5fb-1a83-42b3-b733-6dc686c9cfda None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:38.049613 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ca5fb-1a83-42b3-b733-6dc686c9cfda None None] PATH_INFO: `/` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:38.051401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1167/2273] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 11:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:58:38.058836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bc2f2-a596-46a3-8645-fd586fd2b8dc None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:38.059543 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bc2f2-a596-46a3-8645-fd586fd2b8dc None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:38.059836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bc2f2-a596-46a3-8645-fd586fd2b8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:38.085211 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a63bc2f2-a596-46a3-8645-fd586fd2b8dc None None] MFA Rules not processed for user `5f095d3370be457cb6687ac8f8c9f8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:38.086593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a63bc2f2-a596-46a3-8645-fd586fd2b8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:38.141974 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1107/2274] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 11:58:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:38.154336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ca5fb-1a83-42b3-b733-6dc686c9cfda None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:38.156238 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fb9ca5fb-1a83-42b3-b733-6dc686c9cfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:38.170253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ca5fb-1a83-42b3-b733-6dc686c9cfda None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:38.174223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1a0ab-2786-40f1-b61e-4d9d0c8ac45c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:38.174873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1a0ab-2786-40f1-b61e-4d9d0c8ac45c service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:38.175166 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1a0ab-2786-40f1-b61e-4d9d0c8ac45c service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:38.175448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1a0ab-2786-40f1-b61e-4d9d0c8ac45c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:38.183098 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1a0ab-2786-40f1-b61e-4d9d0c8ac45c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:38.184944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1a0ab-2786-40f1-b61e-4d9d0c8ac45c service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:38.193472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1168/2275] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:58:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:58:38.283231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d8e8f-53db-44d7-b88f-a99f4a5cd2d6 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:38.283638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d8e8f-53db-44d7-b88f-a99f4a5cd2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:38.284387 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d8e8f-53db-44d7-b88f-a99f4a5cd2d6 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:38.285458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1108/2276] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 11:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 11:58:38.290958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e6962-1c6c-40ae-8a03-3fbb913c558d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:38.291249 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e6962-1c6c-40ae-8a03-3fbb913c558d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:38.291528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e6962-1c6c-40ae-8a03-3fbb913c558d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:38.311028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b23e6962-1c6c-40ae-8a03-3fbb913c558d None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:58:38.311673 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b23e6962-1c6c-40ae-8a03-3fbb913c558d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:58:38.321097 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1169/2277] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 11:58:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:58:38.331967 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d8e8f-53db-44d7-b88f-a99f4a5cd2d6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:58:38.344401 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d8e8f-53db-44d7-b88f-a99f4a5cd2d6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:58:38.349827 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03a741-84d4-4d20-b323-74a7a8f490ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:58:38.350634 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03a741-84d4-4d20-b323-74a7a8f490ff service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:58:38.350722 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03a741-84d4-4d20-b323-74a7a8f490ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:58:38.351009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03a741-84d4-4d20-b323-74a7a8f490ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:58:38.370274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03a741-84d4-4d20-b323-74a7a8f490ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:58:38.371927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad03a741-84d4-4d20-b323-74a7a8f490ff service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:58:38.384896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1109/2278] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:58:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:58:48.304991 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=55 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:48.379105 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=35 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:58:59.878983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e6962-1c6c-40ae-8a03-3fbb913c558d None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:00.008461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23e6962-1c6c-40ae-8a03-3fbb913c558d None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:00.014236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:00.015489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:00.015883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:00.016307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] PATH_INFO: `/v3/users/85138dda70fd4c4697289e5bfd137010` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:00.017787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=85138dda70fd4c4697289e5bfd137010)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:00.019918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:00.080790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85138dda70fd4c4697289e5bfd137010'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:00.081457 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85138dda70fd4c4697289e5bfd137010'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:00.082280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85138dda70fd4c4697289e5bfd137010'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:00.095922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85138dda70fd4c4697289e5bfd137010'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:00.105166 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85138dda70fd4c4697289e5bfd137010'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:00.130016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1170/2279] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:58:59 2022] DELETE /identity/v3/users/85138dda70fd4c4697289e5bfd137010 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:59:00.134681 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e550c7-9c93-4024-b9bc-6fac09b47bc1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:00.135261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e550c7-9c93-4024-b9bc-6fac09b47bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:00.135261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e550c7-9c93-4024-b9bc-6fac09b47bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:00.160001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-41e550c7-9c93-4024-b9bc-6fac09b47bc1 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:00.161871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-41e550c7-9c93-4024-b9bc-6fac09b47bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:00.245833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1110/2280] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:00.256627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:00.258431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:00.325105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f631ec-e687-4b0d-ba4c-a715c88e5abd admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:00.329561 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4437cda-f924-46e3-b15e-291fc34e496d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:00.330258 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4437cda-f924-46e3-b15e-291fc34e496d service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:00.330549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4437cda-f924-46e3-b15e-291fc34e496d service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:00.330835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4437cda-f924-46e3-b15e-291fc34e496d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:00.341735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4437cda-f924-46e3-b15e-291fc34e496d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:00.343248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4437cda-f924-46e3-b15e-291fc34e496d service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:00.354314 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1171/2281] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:59:01.315415 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b77839-9ecf-4a04-8480-8cefb08f6d76 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:01.315897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b77839-9ecf-4a04-8480-8cefb08f6d76 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:01.316156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b77839-9ecf-4a04-8480-8cefb08f6d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:01.340326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-11b77839-9ecf-4a04-8480-8cefb08f6d76 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:01.342977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-11b77839-9ecf-4a04-8480-8cefb08f6d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:01.429605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4437cda-f924-46e3-b15e-291fc34e496d service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:01.432120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d4437cda-f924-46e3-b15e-291fc34e496d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:01.447893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1111/2282] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:59:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:01.471264 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4437cda-f924-46e3-b15e-291fc34e496d service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:01.484395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b77839-9ecf-4a04-8480-8cefb08f6d76 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:01.486006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:01.487048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:01.487359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:01.487729 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457 admin admin] PATH_INFO: `/v3/projects/e5e31b080963402f8f2553fbc16e4311` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:01.489263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e31b080963402f8f2553fbc16e4311)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:01.492163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:01.493265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-11b77839-9ecf-4a04-8480-8cefb08f6d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:01.533828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:01.608258 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e31b080963402f8f2553fbc16e4311', 'request_id': 'req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:01.620896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e31b080963402f8f2553fbc16e4311', 'request_id': 'req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:01.628290 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e31b080963402f8f2553fbc16e4311', 'request_id': 'req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:01.645577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457 admin admin] The token cache is being invalidate because project e5e31b080963402f8f2553fbc16e4311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:59:01.646074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:01.647749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-06ad1e6a-abb5-43c0-8dd8-e6510eefc457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:01.650549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1172/2283] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:59:01 2022] DELETE /identity/v3/projects/e5e31b080963402f8f2553fbc16e4311 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:59:01.655304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ec3f5-8580-4438-8a73-2da0645bc993 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:01.660966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ec3f5-8580-4438-8a73-2da0645bc993 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:01.660966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0ec3f5-8580-4438-8a73-2da0645bc993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:01.670299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b77839-9ecf-4a04-8480-8cefb08f6d76 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:01.686754 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9e0ec3f5-8580-4438-8a73-2da0645bc993 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:01.687993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9e0ec3f5-8580-4438-8a73-2da0645bc993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:01.732148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:01.751362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c9a21-e87d-4b08-b256-fbdcca447c10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:01.752046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c9a21-e87d-4b08-b256-fbdcca447c10 service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:01.752338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c9a21-e87d-4b08-b256-fbdcca447c10 service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:01.752624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c9a21-e87d-4b08-b256-fbdcca447c10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:01.754662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-911c9a21-e87d-4b08-b256-fbdcca447c10 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:01.756780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1173/2284] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:01.826460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c9a21-e87d-4b08-b256-fbdcca447c10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:01.826607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198aef9-fc18-48cd-a9da-da3e1d44ba91 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:01.826792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198aef9-fc18-48cd-a9da-da3e1d44ba91 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:01.827048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198aef9-fc18-48cd-a9da-da3e1d44ba91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:01.828193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911c9a21-e87d-4b08-b256-fbdcca447c10 service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:01.836521 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1112/2285] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:59:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:59:01.846655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7198aef9-fc18-48cd-a9da-da3e1d44ba91 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:01.847882 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7198aef9-fc18-48cd-a9da-da3e1d44ba91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:01.859350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1174/2286] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:59:01.862297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c9a21-e87d-4b08-b256-fbdcca447c10 service swift] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:01.874492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911c9a21-e87d-4b08-b256-fbdcca447c10 service swift] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:01.881517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ed076-f451-4202-8428-9b42f4e148ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:01.882338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ed076-f451-4202-8428-9b42f4e148ca admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:01.882733 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ed076-f451-4202-8428-9b42f4e148ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:01.883339 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6ed076-f451-4202-8428-9b42f4e148ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:01.884574 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ed076-f451-4202-8428-9b42f4e148ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:59:01.885039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ed076-f451-4202-8428-9b42f4e148ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:01.889809 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6ed076-f451-4202-8428-9b42f4e148ca admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:01.909082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1113/2287] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:59:01.912048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198aef9-fc18-48cd-a9da-da3e1d44ba91 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:01.925163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198aef9-fc18-48cd-a9da-da3e1d44ba91 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:01.931510 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8b8cd-63d9-408f-b6c1-33f38f83f45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:01.932415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8b8cd-63d9-408f-b6c1-33f38f83f45b admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:01.932580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8b8cd-63d9-408f-b6c1-33f38f83f45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:01.933032 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8b8cd-63d9-408f-b6c1-33f38f83f45b admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:01.933810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8b8cd-63d9-408f-b6c1-33f38f83f45b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:01.935816 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8b8cd-63d9-408f-b6c1-33f38f83f45b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:01.980032 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1175/2288] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:59:01 2022] POST /identity/v3/projects => generated 389 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:59:01.980702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ed076-f451-4202-8428-9b42f4e148ca admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:01.994249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6ed076-f451-4202-8428-9b42f4e148ca admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:02.215991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0ec35-f5ee-47f8-881c-8cbbd6d02698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:02.216836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0ec35-f5ee-47f8-881c-8cbbd6d02698 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:02.228650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0ec35-f5ee-47f8-881c-8cbbd6d02698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:02.228966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0ec35-f5ee-47f8-881c-8cbbd6d02698 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:02.229720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0ec35-f5ee-47f8-881c-8cbbd6d02698 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:02.238192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0ec35-f5ee-47f8-881c-8cbbd6d02698 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:02.284612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1114/2289] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:59:01 2022] POST /identity/v3/users => generated 295 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:59:02.286013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8b8cd-63d9-408f-b6c1-33f38f83f45b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:02.299253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8b8cd-63d9-408f-b6c1-33f38f83f45b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:02.305552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e50b74-9080-4ff2-bb71-9c4c8f544c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:02.307368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e50b74-9080-4ff2-bb71-9c4c8f544c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:02.308108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e50b74-9080-4ff2-bb71-9c4c8f544c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:02.308537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e50b74-9080-4ff2-bb71-9c4c8f544c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:02.309585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e50b74-9080-4ff2-bb71-9c4c8f544c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:02.311664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e50b74-9080-4ff2-bb71-9c4c8f544c59 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:02.328428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0ec35-f5ee-47f8-881c-8cbbd6d02698 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:02.329972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1176/2290] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:59:02.342899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0ec35-f5ee-47f8-881c-8cbbd6d02698 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:02.349413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b597bf2-23b8-4736-aad4-002ea8390f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:02.350135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b597bf2-23b8-4736-aad4-002ea8390f37 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:02.350420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b597bf2-23b8-4736-aad4-002ea8390f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:02.350827 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b597bf2-23b8-4736-aad4-002ea8390f37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:02.352079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b597bf2-23b8-4736-aad4-002ea8390f37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:02.353629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b597bf2-23b8-4736-aad4-002ea8390f37 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:02.368033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1115/2291] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:59:02.369343 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e50b74-9080-4ff2-bb71-9c4c8f544c59 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:02.388405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e50b74-9080-4ff2-bb71-9c4c8f544c59 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:02.393735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7946c8a-f0e2-4bb3-937e-674d099e61b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:02.394936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7946c8a-f0e2-4bb3-937e-674d099e61b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:02.394936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7946c8a-f0e2-4bb3-937e-674d099e61b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:02.394936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7946c8a-f0e2-4bb3-937e-674d099e61b4 admin admin] PATH_INFO: `/v3/projects/1703977bb9f44215b6bc890e36f92926/users/e5242151ea864f75bea16d5c92791a1d/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:02.424131 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7946c8a-f0e2-4bb3-937e-674d099e61b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1703977bb9f44215b6bc890e36f92926, user_id=e5242151ea864f75bea16d5c92791a1d, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:02.435427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7946c8a-f0e2-4bb3-937e-674d099e61b4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:02.475021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1177/2292] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:59:02 2022] PUT /identity/v3/projects/1703977bb9f44215b6bc890e36f92926/users/e5242151ea864f75bea16d5c92791a1d/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:59:02.491335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ad047-bc9b-42c9-9960-36d8d126fd0d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:02.491335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ad047-bc9b-42c9-9960-36d8d126fd0d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:02.491335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ad047-bc9b-42c9-9960-36d8d126fd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:02.538000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-fc8ad047-bc9b-42c9-9960-36d8d126fd0d None None] MFA Rules not processed for user `e5242151ea864f75bea16d5c92791a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:02.543569 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fc8ad047-bc9b-42c9-9960-36d8d126fd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:02.680283 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:02.687991 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176462-a7a9-4fc3-bde8-7d0d36faade2 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:02.688290 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176462-a7a9-4fc3-bde8-7d0d36faade2 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:02.688576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176462-a7a9-4fc3-bde8-7d0d36faade2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:02.713622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f5176462-a7a9-4fc3-bde8-7d0d36faade2 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:02.714878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f5176462-a7a9-4fc3-bde8-7d0d36faade2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:02.746481 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1116/2293] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:02.754818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ad047-bc9b-42c9-9960-36d8d126fd0d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:02.756271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-fc8ad047-bc9b-42c9-9960-36d8d126fd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:02.776745 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:02.810730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1178/2294] 10.222.0.67 () {58 vars in 967 bytes} [Thu Jun 23 11:59:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:59:02.827092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176462-a7a9-4fc3-bde8-7d0d36faade2 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:02.828581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f5176462-a7a9-4fc3-bde8-7d0d36faade2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:02.860910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ad047-bc9b-42c9-9960-36d8d126fd0d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:02.869214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87e9f9-0c24-4dff-a5c8-843bbefd3c36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:02.869883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87e9f9-0c24-4dff-a5c8-843bbefd3c36 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:02.870173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87e9f9-0c24-4dff-a5c8-843bbefd3c36 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:02.870460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87e9f9-0c24-4dff-a5c8-843bbefd3c36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:02.884971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87e9f9-0c24-4dff-a5c8-843bbefd3c36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:02.886919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87e9f9-0c24-4dff-a5c8-843bbefd3c36 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:02.896715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1117/2295] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 11:59:02.933432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176462-a7a9-4fc3-bde8-7d0d36faade2 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:02.941678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76196ef1-e9b7-4f32-86b7-e63e96124dfa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:02.942373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76196ef1-e9b7-4f32-86b7-e63e96124dfa service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:02.942660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76196ef1-e9b7-4f32-86b7-e63e96124dfa service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:02.943030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76196ef1-e9b7-4f32-86b7-e63e96124dfa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:02.952495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76196ef1-e9b7-4f32-86b7-e63e96124dfa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:02.954303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76196ef1-e9b7-4f32-86b7-e63e96124dfa service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:02.980763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1179/2296] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:59:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 11:59:03.250672 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7416e9d-9509-4800-b2a9-af6b8d4ee76e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:03.251767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7416e9d-9509-4800-b2a9-af6b8d4ee76e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:03.252837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7416e9d-9509-4800-b2a9-af6b8d4ee76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:03.281437 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e7416e9d-9509-4800-b2a9-af6b8d4ee76e None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:03.282692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e7416e9d-9509-4800-b2a9-af6b8d4ee76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:03.297238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1118/2297] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:59:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:03.304761 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76196ef1-e9b7-4f32-86b7-e63e96124dfa service swift] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:03.317715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76196ef1-e9b7-4f32-86b7-e63e96124dfa service swift] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:03.325051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9103a-ceb6-4032-87a6-270d96fea0fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:03.326171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9103a-ceb6-4032-87a6-270d96fea0fc service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:03.326575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9103a-ceb6-4032-87a6-270d96fea0fc service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:03.326999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9103a-ceb6-4032-87a6-270d96fea0fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:03.336702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9103a-ceb6-4032-87a6-270d96fea0fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:03.339193 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9103a-ceb6-4032-87a6-270d96fea0fc service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:03.349178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1180/2298] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:59:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:59:03.664727 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8c2ae-3f50-431f-83c6-2089cc94378d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:03.665200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8c2ae-3f50-431f-83c6-2089cc94378d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:03.665584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8c2ae-3f50-431f-83c6-2089cc94378d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:03.687207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8ab8c2ae-3f50-431f-83c6-2089cc94378d None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:03.689001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8ab8c2ae-3f50-431f-83c6-2089cc94378d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:03.691368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc236b-91ca-4ee4-affe-2314e2165ce9 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:03.692242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc236b-91ca-4ee4-affe-2314e2165ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:03.692918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fc236b-91ca-4ee4-affe-2314e2165ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:03.702603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1119/2299] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 11:59:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:03.710347 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8c2ae-3f50-431f-83c6-2089cc94378d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:03.723261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8c2ae-3f50-431f-83c6-2089cc94378d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:03.723383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-84fc236b-91ca-4ee4-affe-2314e2165ce9 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:03.724539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-84fc236b-91ca-4ee4-affe-2314e2165ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:03.731391 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4ebb4-0ee8-4cd9-9a9b-9d4b1f1868cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:03.732165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4ebb4-0ee8-4cd9-9a9b-9d4b1f1868cb service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:03.732378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1181/2300] 10.222.0.67 () {58 vars in 967 bytes} [Thu Jun 23 11:59:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:59:03.732618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4ebb4-0ee8-4cd9-9a9b-9d4b1f1868cb service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:03.732995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4ebb4-0ee8-4cd9-9a9b-9d4b1f1868cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:03.744111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4ebb4-0ee8-4cd9-9a9b-9d4b1f1868cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:03.747087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc236b-91ca-4ee4-affe-2314e2165ce9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:03.750366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4ebb4-0ee8-4cd9-9a9b-9d4b1f1868cb service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:03.763833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1120/2301] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:59:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 11:59:03.765508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fc236b-91ca-4ee4-affe-2314e2165ce9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:03.772772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f0455-0e02-4392-aa33-b683839f2280 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:03.773382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f0455-0e02-4392-aa33-b683839f2280 service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:03.773668 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f0455-0e02-4392-aa33-b683839f2280 service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:03.773947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f0455-0e02-4392-aa33-b683839f2280 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:03.784096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f0455-0e02-4392-aa33-b683839f2280 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:03.786501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f0455-0e02-4392-aa33-b683839f2280 service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:03.800885 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1182/2302] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:59:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 11:59:11.925229 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4ebb4-0ee8-4cd9-9a9b-9d4b1f1868cb service swift] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:11.926862 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e7a4ebb4-0ee8-4cd9-9a9b-9d4b1f1868cb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:12.049109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4ebb4-0ee8-4cd9-9a9b-9d4b1f1868cb service swift] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:12.065174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:12.066489 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:12.066795 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:12.067075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] PATH_INFO: `/v3/users/74464ea11cbc4d44999583f93283a411` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:12.068397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=74464ea11cbc4d44999583f93283a411)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:12.070538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:12.138017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:12.143393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74464ea11cbc4d44999583f93283a411'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:12.145560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74464ea11cbc4d44999583f93283a411'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:12.147232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74464ea11cbc4d44999583f93283a411'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:12.181724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74464ea11cbc4d44999583f93283a411'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:12.208575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74464ea11cbc4d44999583f93283a411'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:12.232888 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1121/2303] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:59:11 2022] DELETE /identity/v3/users/74464ea11cbc4d44999583f93283a411 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:59:12.239536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a261b99-0358-4cb4-b0c0-9de75c6756f3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:12.240510 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a261b99-0358-4cb4-b0c0-9de75c6756f3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:12.240510 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a261b99-0358-4cb4-b0c0-9de75c6756f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:12.272898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5a261b99-0358-4cb4-b0c0-9de75c6756f3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:12.275185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5a261b99-0358-4cb4-b0c0-9de75c6756f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:12.398559 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1183/2304] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:12.411517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:12.413724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:12.481365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56fb953-e391-457c-b727-fe412bf738dc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:12.485067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724ddc5-010f-4e67-a9da-932fbab49126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:12.485947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724ddc5-010f-4e67-a9da-932fbab49126 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:12.486178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724ddc5-010f-4e67-a9da-932fbab49126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:12.486623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9724ddc5-010f-4e67-a9da-932fbab49126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:12.497810 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724ddc5-010f-4e67-a9da-932fbab49126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:12.499349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9724ddc5-010f-4e67-a9da-932fbab49126 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:12.510621 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1122/2305] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:59:12.725791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a261b99-0358-4cb4-b0c0-9de75c6756f3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:12.729004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5a261b99-0358-4cb4-b0c0-9de75c6756f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:12.743063 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a261b99-0358-4cb4-b0c0-9de75c6756f3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:12.748498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:12.749461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:12.749813 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:12.750160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] PATH_INFO: `/v3/projects/d410564d1ba94f7883cd07fcf16be765` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:12.751447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d410564d1ba94f7883cd07fcf16be765)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:12.753650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:12.830129 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:12.838781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd410564d1ba94f7883cd07fcf16be765', 'request_id': 'req-6da7ba8f-04d5-48d5-842a-421abea33e48'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:12.862051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd410564d1ba94f7883cd07fcf16be765', 'request_id': 'req-6da7ba8f-04d5-48d5-842a-421abea33e48'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:12.876038 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd410564d1ba94f7883cd07fcf16be765', 'request_id': 'req-6da7ba8f-04d5-48d5-842a-421abea33e48'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:12.903285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] The token cache is being invalidate because project d410564d1ba94f7883cd07fcf16be765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:59:12.903522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:12.904250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:12.906089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1184/2306] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:59:12 2022] DELETE /identity/v3/projects/d410564d1ba94f7883cd07fcf16be765 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:59:22.505240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=4,keystone:SELECT=39,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:22.898544 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:23.790480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724ddc5-010f-4e67-a9da-932fbab49126 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:23.800017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9724ddc5-010f-4e67-a9da-932fbab49126 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:23.848328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049b425-77fa-46f7-8fdd-b7ad2d35b3c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:23.849129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049b425-77fa-46f7-8fdd-b7ad2d35b3c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:23.849421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049b425-77fa-46f7-8fdd-b7ad2d35b3c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:23.849700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049b425-77fa-46f7-8fdd-b7ad2d35b3c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:23.851523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1049b425-77fa-46f7-8fdd-b7ad2d35b3c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:23.927630 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049b425-77fa-46f7-8fdd-b7ad2d35b3c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:23.929118 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049b425-77fa-46f7-8fdd-b7ad2d35b3c6 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:23.939290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1123/2307] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:59:26.737680 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:26.740688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:26.827225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7ba8f-04d5-48d5-842a-421abea33e48 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:26.832653 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:26.833746 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:26.833914 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:26.834487 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] PATH_INFO: `/v3/users/e5242151ea864f75bea16d5c92791a1d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:26.836164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5242151ea864f75bea16d5c92791a1d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:26.838728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:26.922925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5242151ea864f75bea16d5c92791a1d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:26.923620 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5242151ea864f75bea16d5c92791a1d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:26.924248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5242151ea864f75bea16d5c92791a1d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:26.942195 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5242151ea864f75bea16d5c92791a1d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:26.951900 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5242151ea864f75bea16d5c92791a1d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:26.974782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1185/2308] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:59:26 2022] DELETE /identity/v3/users/e5242151ea864f75bea16d5c92791a1d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:59:26.978967 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed75697-dbd6-4826-b6b6-5f18f6b85bbf None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:26.979418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed75697-dbd6-4826-b6b6-5f18f6b85bbf None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:26.979692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed75697-dbd6-4826-b6b6-5f18f6b85bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:27.007683 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-eed75697-dbd6-4826-b6b6-5f18f6b85bbf None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:27.009257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-eed75697-dbd6-4826-b6b6-5f18f6b85bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:27.119730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1124/2309] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:27.133077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:27.134857 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:27.219194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc1508-1fe8-4148-9042-64605c36ca4d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:27.222832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8717e9-e4ae-4884-87bc-b8ac7c61fc54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:27.223479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8717e9-e4ae-4884-87bc-b8ac7c61fc54 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:27.223767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8717e9-e4ae-4884-87bc-b8ac7c61fc54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:27.224050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8717e9-e4ae-4884-87bc-b8ac7c61fc54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:27.234288 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8717e9-e4ae-4884-87bc-b8ac7c61fc54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:27.235767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8717e9-e4ae-4884-87bc-b8ac7c61fc54 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:27.247167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1186/2310] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:59:27.953016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed75697-dbd6-4826-b6b6-5f18f6b85bbf None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:27.954523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-eed75697-dbd6-4826-b6b6-5f18f6b85bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:27.971693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed75697-dbd6-4826-b6b6-5f18f6b85bbf None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:27.977901 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bb94d-a061-40ba-8758-22623dbc1230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:27.979178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bb94d-a061-40ba-8758-22623dbc1230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:27.981971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bb94d-a061-40ba-8758-22623dbc1230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:27.982415 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bb94d-a061-40ba-8758-22623dbc1230 admin admin] PATH_INFO: `/v3/projects/1703977bb9f44215b6bc890e36f92926` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:27.984475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bb94d-a061-40ba-8758-22623dbc1230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1703977bb9f44215b6bc890e36f92926)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:27.995280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1bb94d-a061-40ba-8758-22623dbc1230 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:28.080979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:28.085195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cc1bb94d-a061-40ba-8758-22623dbc1230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1703977bb9f44215b6bc890e36f92926', 'request_id': 'req-cc1bb94d-a061-40ba-8758-22623dbc1230'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:28.098167 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cc1bb94d-a061-40ba-8758-22623dbc1230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1703977bb9f44215b6bc890e36f92926', 'request_id': 'req-cc1bb94d-a061-40ba-8758-22623dbc1230'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:28.106197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cc1bb94d-a061-40ba-8758-22623dbc1230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1703977bb9f44215b6bc890e36f92926', 'request_id': 'req-cc1bb94d-a061-40ba-8758-22623dbc1230'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:28.120112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cc1bb94d-a061-40ba-8758-22623dbc1230 admin admin] The token cache is being invalidate because project 1703977bb9f44215b6bc890e36f92926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:59:28.120416 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cc1bb94d-a061-40ba-8758-22623dbc1230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:28.121829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cc1bb94d-a061-40ba-8758-22623dbc1230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:28.123480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1125/2311] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:59:27 2022] DELETE /identity/v3/projects/1703977bb9f44215b6bc890e36f92926 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:59:30.863469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1faa077-8c8c-4eb2-b1d2-0e54737b8b55 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:30.864265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1faa077-8c8c-4eb2-b1d2-0e54737b8b55 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:30.864265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1faa077-8c8c-4eb2-b1d2-0e54737b8b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:30.882569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e1faa077-8c8c-4eb2-b1d2-0e54737b8b55 None None] MFA Rules not processed for user `53f1188f2446493baae7dc79bbb66ff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:30.883354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e1faa077-8c8c-4eb2-b1d2-0e54737b8b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:30.901222 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:30.944682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1187/2312] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:31.872698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9aab59-2526-4e39-ae06-70e33b0d3b03 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:31.873200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9aab59-2526-4e39-ae06-70e33b0d3b03 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:31.873296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9aab59-2526-4e39-ae06-70e33b0d3b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:31.891460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7b9aab59-2526-4e39-ae06-70e33b0d3b03 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:31.892698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7b9aab59-2526-4e39-ae06-70e33b0d3b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:31.950314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1126/2313] 10.222.0.67 () {58 vars in 967 bytes} [Thu Jun 23 11:59:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:31.962554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1faa077-8c8c-4eb2-b1d2-0e54737b8b55 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:31.964107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e1faa077-8c8c-4eb2-b1d2-0e54737b8b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:32.035309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1faa077-8c8c-4eb2-b1d2-0e54737b8b55 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:32.040167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0af090-d2a5-4215-92f2-a052fc440df4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:32.040839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0af090-d2a5-4215-92f2-a052fc440df4 service swift] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:32.041122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0af090-d2a5-4215-92f2-a052fc440df4 service swift] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:32.041395 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0af090-d2a5-4215-92f2-a052fc440df4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:32.050311 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0af090-d2a5-4215-92f2-a052fc440df4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:32.051893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0af090-d2a5-4215-92f2-a052fc440df4 service swift] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:32.060755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1188/2314] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 11:59:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 11:59:33.983500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9aab59-2526-4e39-ae06-70e33b0d3b03 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:33.985131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7b9aab59-2526-4e39-ae06-70e33b0d3b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:34.051529 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9aab59-2526-4e39-ae06-70e33b0d3b03 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:34.056039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb83a8a-9600-4c6e-88f5-d26125f0f53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:34.056937 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb83a8a-9600-4c6e-88f5-d26125f0f53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:34.057236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb83a8a-9600-4c6e-88f5-d26125f0f53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:34.057519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb83a8a-9600-4c6e-88f5-d26125f0f53e admin admin] PATH_INFO: `/v3/users/f7cf8ed5de4b41b19b6f25299087e1b5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:34.059469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb83a8a-9600-4c6e-88f5-d26125f0f53e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7cf8ed5de4b41b19b6f25299087e1b5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:34.061453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb83a8a-9600-4c6e-88f5-d26125f0f53e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:34.107795 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3fb83a8a-9600-4c6e-88f5-d26125f0f53e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7cf8ed5de4b41b19b6f25299087e1b5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:34.108469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3fb83a8a-9600-4c6e-88f5-d26125f0f53e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7cf8ed5de4b41b19b6f25299087e1b5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:34.109110 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3fb83a8a-9600-4c6e-88f5-d26125f0f53e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7cf8ed5de4b41b19b6f25299087e1b5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:34.121378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3fb83a8a-9600-4c6e-88f5-d26125f0f53e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7cf8ed5de4b41b19b6f25299087e1b5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:34.128231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3fb83a8a-9600-4c6e-88f5-d26125f0f53e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7cf8ed5de4b41b19b6f25299087e1b5'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:34.137074 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:34.143633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1127/2315] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 11:59:33 2022] DELETE /identity/v3/users/f7cf8ed5de4b41b19b6f25299087e1b5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:59:34.319943 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0af090-d2a5-4215-92f2-a052fc440df4 service swift] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:34.331496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0af090-d2a5-4215-92f2-a052fc440df4 service swift] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:34.392603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9544bed-3b6c-470f-9d17-95f786318491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:34.393496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9544bed-3b6c-470f-9d17-95f786318491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:34.393786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9544bed-3b6c-470f-9d17-95f786318491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:34.394069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9544bed-3b6c-470f-9d17-95f786318491 admin admin] PATH_INFO: `/v3/projects/a5a0086895b6442da90bc0f1bcecd3e7` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:34.395399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9544bed-3b6c-470f-9d17-95f786318491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a0086895b6442da90bc0f1bcecd3e7)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:34.397363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9544bed-3b6c-470f-9d17-95f786318491 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:34.443697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a9544bed-3b6c-470f-9d17-95f786318491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a0086895b6442da90bc0f1bcecd3e7', 'request_id': 'req-a9544bed-3b6c-470f-9d17-95f786318491'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:34.450821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a9544bed-3b6c-470f-9d17-95f786318491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a0086895b6442da90bc0f1bcecd3e7', 'request_id': 'req-a9544bed-3b6c-470f-9d17-95f786318491'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:34.452889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:34.457967 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a9544bed-3b6c-470f-9d17-95f786318491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a0086895b6442da90bc0f1bcecd3e7', 'request_id': 'req-a9544bed-3b6c-470f-9d17-95f786318491'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:34.470081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a9544bed-3b6c-470f-9d17-95f786318491 admin admin] The token cache is being invalidate because project a5a0086895b6442da90bc0f1bcecd3e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 11:59:34.470978 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a9544bed-3b6c-470f-9d17-95f786318491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:34.471637 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a9544bed-3b6c-470f-9d17-95f786318491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 11:59:34.473223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1189/2316] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 11:59:34 2022] DELETE /identity/v3/projects/a5a0086895b6442da90bc0f1bcecd3e7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:59:34.478162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a229e6-fb94-4294-83b7-e81da660d788 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:34.478451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a229e6-fb94-4294-83b7-e81da660d788 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:34.478729 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a229e6-fb94-4294-83b7-e81da660d788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:34.497655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-72a229e6-fb94-4294-83b7-e81da660d788 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:34.498858 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-72a229e6-fb94-4294-83b7-e81da660d788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:34.557572 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1128/2317] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:34.565929 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ed72f-0d02-4118-8c28-5ab6b1d899bd None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:34.566285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ed72f-0d02-4118-8c28-5ab6b1d899bd None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:34.566575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ed72f-0d02-4118-8c28-5ab6b1d899bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:34.584770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ef2ed72f-0d02-4118-8c28-5ab6b1d899bd None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:34.586005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ef2ed72f-0d02-4118-8c28-5ab6b1d899bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:34.593956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1190/2318] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:34.598104 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a229e6-fb94-4294-83b7-e81da660d788 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:34.608272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a229e6-fb94-4294-83b7-e81da660d788 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:34.612917 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc25137-46d5-4625-b80b-0fc5e47985d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:34.613564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc25137-46d5-4625-b80b-0fc5e47985d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:34.613848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc25137-46d5-4625-b80b-0fc5e47985d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:34.614122 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc25137-46d5-4625-b80b-0fc5e47985d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:34.614892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc25137-46d5-4625-b80b-0fc5e47985d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 11:59:34.615200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc25137-46d5-4625-b80b-0fc5e47985d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:34.616820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc25137-46d5-4625-b80b-0fc5e47985d2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:34.629086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1129/2319] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 11:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 11:59:34.633363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ed72f-0d02-4118-8c28-5ab6b1d899bd None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:34.642763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ed72f-0d02-4118-8c28-5ab6b1d899bd None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:34.647772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110b508-b0e4-4099-947d-3c3b0aa1d17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:34.648436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110b508-b0e4-4099-947d-3c3b0aa1d17a admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:34.648829 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110b508-b0e4-4099-947d-3c3b0aa1d17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:34.649482 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110b508-b0e4-4099-947d-3c3b0aa1d17a admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:34.649832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110b508-b0e4-4099-947d-3c3b0aa1d17a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:34.651760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110b508-b0e4-4099-947d-3c3b0aa1d17a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:34.687330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1191/2320] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:59:34 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:59:34.691779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc25137-46d5-4625-b80b-0fc5e47985d2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:34.702077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc25137-46d5-4625-b80b-0fc5e47985d2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:34.759670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e235fe-7418-4e75-926e-f2118baef374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:34.760183 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e235fe-7418-4e75-926e-f2118baef374 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:34.760475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e235fe-7418-4e75-926e-f2118baef374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:34.760789 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e235fe-7418-4e75-926e-f2118baef374 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:34.761526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e235fe-7418-4e75-926e-f2118baef374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:34.763440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e235fe-7418-4e75-926e-f2118baef374 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:34.784779 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1130/2321] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:59:34 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:59:34.788968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110b508-b0e4-4099-947d-3c3b0aa1d17a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:34.800442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110b508-b0e4-4099-947d-3c3b0aa1d17a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:34.810755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2327340-91ce-4cfa-a177-bc61bfc007e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:34.811977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2327340-91ce-4cfa-a177-bc61bfc007e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:34.811977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2327340-91ce-4cfa-a177-bc61bfc007e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:34.812251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2327340-91ce-4cfa-a177-bc61bfc007e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:34.813094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2327340-91ce-4cfa-a177-bc61bfc007e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:34.814793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2327340-91ce-4cfa-a177-bc61bfc007e5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:34.826902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1192/2322] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:59:34.830980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e235fe-7418-4e75-926e-f2118baef374 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:34.842297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e235fe-7418-4e75-926e-f2118baef374 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:34.847061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec99e92-5f02-4b8e-900a-dd8a83d47d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:34.848113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec99e92-5f02-4b8e-900a-dd8a83d47d2e admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:34.848113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec99e92-5f02-4b8e-900a-dd8a83d47d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:34.848349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec99e92-5f02-4b8e-900a-dd8a83d47d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:34.849032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec99e92-5f02-4b8e-900a-dd8a83d47d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:34.850631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec99e92-5f02-4b8e-900a-dd8a83d47d2e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:34.862062 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1131/2323] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:59:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 11:59:34.865969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2327340-91ce-4cfa-a177-bc61bfc007e5 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:34.876786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2327340-91ce-4cfa-a177-bc61bfc007e5 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:34.881588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfc3ab-738d-4eea-a12a-faf48adb1858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:34.882472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfc3ab-738d-4eea-a12a-faf48adb1858 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:34.882793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfc3ab-738d-4eea-a12a-faf48adb1858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:34.883078 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cfc3ab-738d-4eea-a12a-faf48adb1858 admin admin] PATH_INFO: `/v3/projects/a2d3e44f4113405a9cd4e604aea9955b/users/62766b6eeb3d47beb5e3d056fb6d5d81/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:34.899769 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cfc3ab-738d-4eea-a12a-faf48adb1858 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2d3e44f4113405a9cd4e604aea9955b, user_id=62766b6eeb3d47beb5e3d056fb6d5d81, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:34.903101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cfc3ab-738d-4eea-a12a-faf48adb1858 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:34.916840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1193/2324] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:59:34 2022] PUT /identity/v3/projects/a2d3e44f4113405a9cd4e604aea9955b/users/62766b6eeb3d47beb5e3d056fb6d5d81/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 11:59:34.922901 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16253d9-111d-472b-8e50-74362b1973b1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:34.923250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16253d9-111d-472b-8e50-74362b1973b1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:34.923539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16253d9-111d-472b-8e50-74362b1973b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:34.941289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c16253d9-111d-472b-8e50-74362b1973b1 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:34.942492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c16253d9-111d-472b-8e50-74362b1973b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:35.001223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1132/2325] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 11:59:35.011681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfc3ab-738d-4eea-a12a-faf48adb1858 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:35.013500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c9cfc3ab-738d-4eea-a12a-faf48adb1858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:35.071348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cfc3ab-738d-4eea-a12a-faf48adb1858 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:35.075169 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e79d9-ded4-401a-80d9-3c6b693a9f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:35.075812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e79d9-ded4-401a-80d9-3c6b693a9f81 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:35.076103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e79d9-ded4-401a-80d9-3c6b693a9f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:35.076384 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e79d9-ded4-401a-80d9-3c6b693a9f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:35.085630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e79d9-ded4-401a-80d9-3c6b693a9f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:35.087051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e79d9-ded4-401a-80d9-3c6b693a9f81 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:35.097043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1194/2326] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:59:39.685123 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554a2c-5ffc-4790-a6a3-692b5632ac09 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:39.686001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554a2c-5ffc-4790-a6a3-692b5632ac09 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:39.686353 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554a2c-5ffc-4790-a6a3-692b5632ac09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:39.693480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:39.717856 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f8554a2c-5ffc-4790-a6a3-692b5632ac09 None None] MFA Rules not processed for user `62766b6eeb3d47beb5e3d056fb6d5d81`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:39.719183 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f8554a2c-5ffc-4790-a6a3-692b5632ac09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:39.789420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1133/2327] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:39.794013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e79d9-ded4-401a-80d9-3c6b693a9f81 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:39.804275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e79d9-ded4-401a-80d9-3c6b693a9f81 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:39.809037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341a1a7-774d-4ccb-aa17-1a55e3c4ac3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:39.809702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341a1a7-774d-4ccb-aa17-1a55e3c4ac3e admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:39.810033 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341a1a7-774d-4ccb-aa17-1a55e3c4ac3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:39.810317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e341a1a7-774d-4ccb-aa17-1a55e3c4ac3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:39.811014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341a1a7-774d-4ccb-aa17-1a55e3c4ac3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:39.812961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e341a1a7-774d-4ccb-aa17-1a55e3c4ac3e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:39.842612 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1195/2328] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 11:59:39 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:59:39.846580 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8554a2c-5ffc-4790-a6a3-692b5632ac09 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:39.858013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8554a2c-5ffc-4790-a6a3-692b5632ac09 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:39.922006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64591e-b4cc-46f7-822a-a5874ed19cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:39.922865 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64591e-b4cc-46f7-822a-a5874ed19cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:39.923148 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64591e-b4cc-46f7-822a-a5874ed19cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:39.923427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64591e-b4cc-46f7-822a-a5874ed19cc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:39.924130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64591e-b4cc-46f7-822a-a5874ed19cc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:39.926101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de64591e-b4cc-46f7-822a-a5874ed19cc2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:39.947610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1134/2329] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 11:59:39 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 11:59:39.952292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341a1a7-774d-4ccb-aa17-1a55e3c4ac3e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:39.963618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e341a1a7-774d-4ccb-aa17-1a55e3c4ac3e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:39.969014 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e321dcd-19cc-434f-aa1d-97c696da2aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:39.969694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e321dcd-19cc-434f-aa1d-97c696da2aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:39.969972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e321dcd-19cc-434f-aa1d-97c696da2aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:39.970342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e321dcd-19cc-434f-aa1d-97c696da2aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:39.970979 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e321dcd-19cc-434f-aa1d-97c696da2aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:39.972591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e321dcd-19cc-434f-aa1d-97c696da2aa2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:39.984084 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1196/2330] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:59:39.988256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64591e-b4cc-46f7-822a-a5874ed19cc2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:39.999091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de64591e-b4cc-46f7-822a-a5874ed19cc2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:40.004266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517c9ea-cafc-402b-b487-829de3452c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:40.005210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517c9ea-cafc-402b-b487-829de3452c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:40.005496 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517c9ea-cafc-402b-b487-829de3452c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:40.005808 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517c9ea-cafc-402b-b487-829de3452c3d admin admin] PATH_INFO: `/v3/projects/971c124aa61549df80a67e641298d55f/users/a5ba399ef99d421cb9870f4932d7379a/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:40.024571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517c9ea-cafc-402b-b487-829de3452c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=971c124aa61549df80a67e641298d55f, user_id=a5ba399ef99d421cb9870f4932d7379a, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:40.027885 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517c9ea-cafc-402b-b487-829de3452c3d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:40.044005 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1135/2331] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:59:39 2022] PUT /identity/v3/projects/971c124aa61549df80a67e641298d55f/users/a5ba399ef99d421cb9870f4932d7379a/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:59:40.048473 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e321dcd-19cc-434f-aa1d-97c696da2aa2 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:40.094209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e321dcd-19cc-434f-aa1d-97c696da2aa2 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:40.162395 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:40.174972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c65c67e-6f57-4728-9c7f-2324e11c7ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:40.175879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65c67e-6f57-4728-9c7f-2324e11c7ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:40.176240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65c67e-6f57-4728-9c7f-2324e11c7ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:40.176597 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c65c67e-6f57-4728-9c7f-2324e11c7ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:40.177443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65c67e-6f57-4728-9c7f-2324e11c7ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:40.179139 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c65c67e-6f57-4728-9c7f-2324e11c7ac1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:40.194207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1197/2332] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 11:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 11:59:40.198087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517c9ea-cafc-402b-b487-829de3452c3d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:40.214640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517c9ea-cafc-402b-b487-829de3452c3d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:40.221582 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd8cbb-2dcc-4681-b53a-94d84ed8da93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:40.222749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd8cbb-2dcc-4681-b53a-94d84ed8da93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:40.222944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd8cbb-2dcc-4681-b53a-94d84ed8da93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:40.223301 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd8cbb-2dcc-4681-b53a-94d84ed8da93 admin admin] PATH_INFO: `/v3/projects/971c124aa61549df80a67e641298d55f/users/a5ba399ef99d421cb9870f4932d7379a/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:40.225664 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd8cbb-2dcc-4681-b53a-94d84ed8da93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=971c124aa61549df80a67e641298d55f, user_id=a5ba399ef99d421cb9870f4932d7379a, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:40.230103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd8cbb-2dcc-4681-b53a-94d84ed8da93 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:40.260437 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1136/2333] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 11:59:40 2022] PUT /identity/v3/projects/971c124aa61549df80a67e641298d55f/users/a5ba399ef99d421cb9870f4932d7379a/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 11:59:45.002360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9292b39-2b6a-44ec-82fa-0a8acc9595ee None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:45.002957 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9292b39-2b6a-44ec-82fa-0a8acc9595ee None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:45.002957 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9292b39-2b6a-44ec-82fa-0a8acc9595ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:45.032436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a9292b39-2b6a-44ec-82fa-0a8acc9595ee None None] MFA Rules not processed for user `a5ba399ef99d421cb9870f4932d7379a`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:45.033684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a9292b39-2b6a-44ec-82fa-0a8acc9595ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:45.112553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1198/2334] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:45 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:45.133029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd8cbb-2dcc-4681-b53a-94d84ed8da93 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:45.134911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e2fd8cbb-2dcc-4681-b53a-94d84ed8da93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:45.189980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 11:59:45.216899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd8cbb-2dcc-4681-b53a-94d84ed8da93 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:45.223157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74199a1c-3723-4feb-9e43-fb3f7cc5ac69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:45.223812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74199a1c-3723-4feb-9e43-fb3f7cc5ac69 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:45.224097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74199a1c-3723-4feb-9e43-fb3f7cc5ac69 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:45.224375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74199a1c-3723-4feb-9e43-fb3f7cc5ac69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:45.284396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74199a1c-3723-4feb-9e43-fb3f7cc5ac69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:45.286061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74199a1c-3723-4feb-9e43-fb3f7cc5ac69 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:45.296456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1137/2335] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:59:46.267094 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179adc-ef82-42c7-9708-8ed10eeb3dee None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:46.267556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179adc-ef82-42c7-9708-8ed10eeb3dee None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:46.267694 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57179adc-ef82-42c7-9708-8ed10eeb3dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:46.286566 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-57179adc-ef82-42c7-9708-8ed10eeb3dee None None] MFA Rules not processed for user `62766b6eeb3d47beb5e3d056fb6d5d81`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 11:59:46.287843 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-57179adc-ef82-42c7-9708-8ed10eeb3dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:46.299429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1199/2336] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 11:59:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 11:59:51.317195 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74199a1c-3723-4feb-9e43-fb3f7cc5ac69 service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:51.393878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74199a1c-3723-4feb-9e43-fb3f7cc5ac69 service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:51.397466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c731fe-842a-4657-9fde-ba03c01766d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:51.398114 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c731fe-842a-4657-9fde-ba03c01766d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:51.398394 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c731fe-842a-4657-9fde-ba03c01766d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:51.398667 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c731fe-842a-4657-9fde-ba03c01766d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:51.400573 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-31c731fe-842a-4657-9fde-ba03c01766d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:51.417239 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c731fe-842a-4657-9fde-ba03c01766d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:51.419551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c731fe-842a-4657-9fde-ba03c01766d0 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:51.431895 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1138/2337] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 11:59:52.479150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57179adc-ef82-42c7-9708-8ed10eeb3dee None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 11:59:52.481126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-57179adc-ef82-42c7-9708-8ed10eeb3dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:52.538388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57179adc-ef82-42c7-9708-8ed10eeb3dee None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 11:59:52.541835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7321a-9047-4134-b8f4-ff0b96f948df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 11:59:52.542476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7321a-9047-4134-b8f4-ff0b96f948df service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 11:59:52.543272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7321a-9047-4134-b8f4-ff0b96f948df service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 11:59:52.543549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7321a-9047-4134-b8f4-ff0b96f948df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 11:59:52.545353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-54d7321a-9047-4134-b8f4-ff0b96f948df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 11:59:52.559263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7321a-9047-4134-b8f4-ff0b96f948df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 11:59:52.560687 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7321a-9047-4134-b8f4-ff0b96f948df service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 11:59:52.572299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1200/2338] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 11:59:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:00.108853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c731fe-842a-4657-9fde-ba03c01766d0 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:00.110443 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-31c731fe-842a-4657-9fde-ba03c01766d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:00.129306 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c731fe-842a-4657-9fde-ba03c01766d0 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:00.135005 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e395d-9a72-43e6-a695-4faa967d5d02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:00.135653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e395d-9a72-43e6-a695-4faa967d5d02 service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:00.135937 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e395d-9a72-43e6-a695-4faa967d5d02 service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:00.136218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e395d-9a72-43e6-a695-4faa967d5d02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:00.138772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-479e395d-9a72-43e6-a695-4faa967d5d02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:00.199023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479e395d-9a72-43e6-a695-4faa967d5d02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:00.201190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479e395d-9a72-43e6-a695-4faa967d5d02 service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:00.210334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1139/2339] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:01.006516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7321a-9047-4134-b8f4-ff0b96f948df service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:01.017652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7321a-9047-4134-b8f4-ff0b96f948df service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:01.023286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6bcf9-da38-4e1e-96a4-61d3f63fa694 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:01.023916 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6bcf9-da38-4e1e-96a4-61d3f63fa694 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:01.024201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6bcf9-da38-4e1e-96a4-61d3f63fa694 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:01.024478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b6bcf9-da38-4e1e-96a4-61d3f63fa694 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:01.026265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-63b6bcf9-da38-4e1e-96a4-61d3f63fa694 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:01.087655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6bcf9-da38-4e1e-96a4-61d3f63fa694 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:01.089316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b6bcf9-da38-4e1e-96a4-61d3f63fa694 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:01.101251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1201/2340] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:02.861151 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e395d-9a72-43e6-a695-4faa967d5d02 service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:02.862554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-479e395d-9a72-43e6-a695-4faa967d5d02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:02.874482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e395d-9a72-43e6-a695-4faa967d5d02 service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:02.878178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761bf58-6c6f-4d5f-bb0d-89103c67527e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:02.878813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761bf58-6c6f-4d5f-bb0d-89103c67527e service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:02.879101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761bf58-6c6f-4d5f-bb0d-89103c67527e service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:02.879383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761bf58-6c6f-4d5f-bb0d-89103c67527e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:02.881141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5761bf58-6c6f-4d5f-bb0d-89103c67527e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:02.890936 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761bf58-6c6f-4d5f-bb0d-89103c67527e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:02.892345 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761bf58-6c6f-4d5f-bb0d-89103c67527e service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:02.900273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1140/2341] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:03.348208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6bcf9-da38-4e1e-96a4-61d3f63fa694 service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:03.349617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-63b6bcf9-da38-4e1e-96a4-61d3f63fa694 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:03.362652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:03.431201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b6bcf9-da38-4e1e-96a4-61d3f63fa694 service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:03.436623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe3aee-7b28-4618-8594-3781f9712fdb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:03.437318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe3aee-7b28-4618-8594-3781f9712fdb service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:03.437605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe3aee-7b28-4618-8594-3781f9712fdb service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:03.437886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe3aee-7b28-4618-8594-3781f9712fdb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:03.439828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8bbe3aee-7b28-4618-8594-3781f9712fdb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:03.453560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe3aee-7b28-4618-8594-3781f9712fdb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:03.455919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe3aee-7b28-4618-8594-3781f9712fdb service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:03.472462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1202/2342] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:06.054855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761bf58-6c6f-4d5f-bb0d-89103c67527e service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:06.078643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761bf58-6c6f-4d5f-bb0d-89103c67527e service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:06.093842 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0240f1-66bf-4377-a8c3-370f2fb4c806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:06.094601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0240f1-66bf-4377-a8c3-370f2fb4c806 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:06.094894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0240f1-66bf-4377-a8c3-370f2fb4c806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:06.095182 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0240f1-66bf-4377-a8c3-370f2fb4c806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:06.097338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-be0240f1-66bf-4377-a8c3-370f2fb4c806 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:06.123121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0240f1-66bf-4377-a8c3-370f2fb4c806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:06.126845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0240f1-66bf-4377-a8c3-370f2fb4c806 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:06.143150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:06.161777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1141/2343] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:08.706036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837a875-c377-4522-8332-468f623337df None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:08.707248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837a875-c377-4522-8332-468f623337df None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:08.707553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837a875-c377-4522-8332-468f623337df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:08.729678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-9837a875-c377-4522-8332-468f623337df None None] MFA Rules not processed for user `9cc891cc8172490394d2bfa3c8d1dcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:08.730919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9837a875-c377-4522-8332-468f623337df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:08.743915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1203/2344] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:00:09.159974 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0240f1-66bf-4377-a8c3-370f2fb4c806 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:09.168583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0240f1-66bf-4377-a8c3-370f2fb4c806 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:09.173077 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc2ec8-92c0-4d6c-84d7-d9637e8a955a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:09.173710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc2ec8-92c0-4d6c-84d7-d9637e8a955a service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:09.173998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc2ec8-92c0-4d6c-84d7-d9637e8a955a service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:09.174305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc2ec8-92c0-4d6c-84d7-d9637e8a955a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:09.176296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2acc2ec8-92c0-4d6c-84d7-d9637e8a955a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:09.186651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc2ec8-92c0-4d6c-84d7-d9637e8a955a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:09.188072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc2ec8-92c0-4d6c-84d7-d9637e8a955a service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:09.196263 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1142/2345] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:12.799898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837a875-c377-4522-8332-468f623337df None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:12.815983 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837a875-c377-4522-8332-468f623337df None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:12.824427 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc4938-f492-4166-96d7-633f2033a09c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:12.825138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc4938-f492-4166-96d7-633f2033a09c service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:12.825421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc4938-f492-4166-96d7-633f2033a09c service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:12.826415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc4938-f492-4166-96d7-633f2033a09c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:12.828309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-15bc4938-f492-4166-96d7-633f2033a09c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:12.841614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc4938-f492-4166-96d7-633f2033a09c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:12.843759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc4938-f492-4166-96d7-633f2033a09c service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:12.859254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1204/2346] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:18.843998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc2ec8-92c0-4d6c-84d7-d9637e8a955a service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:18.845832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2acc2ec8-92c0-4d6c-84d7-d9637e8a955a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:18.879531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc2ec8-92c0-4d6c-84d7-d9637e8a955a service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:18.887813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b01a56-7c11-4035-b553-fbb547bf12a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:18.890542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b01a56-7c11-4035-b553-fbb547bf12a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:18.891834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b01a56-7c11-4035-b553-fbb547bf12a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:18.894705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b01a56-7c11-4035-b553-fbb547bf12a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:18.897694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-00b01a56-7c11-4035-b553-fbb547bf12a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:18.997622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b01a56-7c11-4035-b553-fbb547bf12a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:18.999726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b01a56-7c11-4035-b553-fbb547bf12a8 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:19.011623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1143/2347] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:22.204781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc4938-f492-4166-96d7-633f2033a09c service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:22.217317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc4938-f492-4166-96d7-633f2033a09c service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:22.223165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570389b-616a-4c05-8d7a-1df5dd34031e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:22.223830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570389b-616a-4c05-8d7a-1df5dd34031e service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:22.224118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570389b-616a-4c05-8d7a-1df5dd34031e service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:22.224403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570389b-616a-4c05-8d7a-1df5dd34031e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:22.226360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b570389b-616a-4c05-8d7a-1df5dd34031e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:22.293048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570389b-616a-4c05-8d7a-1df5dd34031e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:22.294778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570389b-616a-4c05-8d7a-1df5dd34031e service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:22.306205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1205/2348] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:23.077543 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b01a56-7c11-4035-b553-fbb547bf12a8 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:23.090665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b01a56-7c11-4035-b553-fbb547bf12a8 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:23.096414 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54e8591-a79b-4368-adbe-e9835ba8577a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:23.097512 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e8591-a79b-4368-adbe-e9835ba8577a service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:23.097991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e8591-a79b-4368-adbe-e9835ba8577a service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:23.098456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54e8591-a79b-4368-adbe-e9835ba8577a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:23.114700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e8591-a79b-4368-adbe-e9835ba8577a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:23.117081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54e8591-a79b-4368-adbe-e9835ba8577a service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:23.131445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1144/2349] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:23 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:25.280397 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570389b-616a-4c05-8d7a-1df5dd34031e service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:25.303593 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570389b-616a-4c05-8d7a-1df5dd34031e service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:25.310836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55817676-df11-4e4d-a7e6-814327db2c09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:25.311516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55817676-df11-4e4d-a7e6-814327db2c09 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:25.311807 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55817676-df11-4e4d-a7e6-814327db2c09 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:25.312116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55817676-df11-4e4d-a7e6-814327db2c09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:25.314151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-55817676-df11-4e4d-a7e6-814327db2c09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:25.329674 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55817676-df11-4e4d-a7e6-814327db2c09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:25.331282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55817676-df11-4e4d-a7e6-814327db2c09 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:25.342614 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1206/2350] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:00:33.123671 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=41 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:35.338187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=71 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:35.533594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ba30f-2eb6-4c6f-9913-fadcb358086a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:35.534158 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ba30f-2eb6-4c6f-9913-fadcb358086a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:35.534158 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ba30f-2eb6-4c6f-9913-fadcb358086a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:35.552617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2a4ba30f-2eb6-4c6f-9913-fadcb358086a None None] MFA Rules not processed for user `ebfb5a6c657a4fd592ff9e67348627eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:35.553870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2a4ba30f-2eb6-4c6f-9913-fadcb358086a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:35.560769 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1145/2351] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:00:44.551380 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55817676-df11-4e4d-a7e6-814327db2c09 service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:44.565255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55817676-df11-4e4d-a7e6-814327db2c09 service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:44.569384 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebcd14-e585-44f1-bf1b-5f7e3c4fc5bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:44.569953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebcd14-e585-44f1-bf1b-5f7e3c4fc5bb service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:44.570233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebcd14-e585-44f1-bf1b-5f7e3c4fc5bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:44.571478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ebcd14-e585-44f1-bf1b-5f7e3c4fc5bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:44.572458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-62ebcd14-e585-44f1-bf1b-5f7e3c4fc5bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:44.661683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebcd14-e585-44f1-bf1b-5f7e3c4fc5bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:44.664492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ebcd14-e585-44f1-bf1b-5f7e3c4fc5bb service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:44.680159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1207/2352] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:45.547695 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:46.917201 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ba30f-2eb6-4c6f-9913-fadcb358086a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:46.936663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ba30f-2eb6-4c6f-9913-fadcb358086a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:46.945034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24228f81-6190-498e-aa79-0229587315ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:46.946502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24228f81-6190-498e-aa79-0229587315ac service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:46.947025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24228f81-6190-498e-aa79-0229587315ac service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:46.947672 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24228f81-6190-498e-aa79-0229587315ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:46.949936 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-24228f81-6190-498e-aa79-0229587315ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:46.966052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24228f81-6190-498e-aa79-0229587315ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:46.967949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24228f81-6190-498e-aa79-0229587315ac service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:46.981551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1146/2353] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:47.935521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebcd14-e585-44f1-bf1b-5f7e3c4fc5bb service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:47.948934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ebcd14-e585-44f1-bf1b-5f7e3c4fc5bb service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:47.957086 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1353c3-3bcb-4ff1-af75-413cc162fb88 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:47.958672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1353c3-3bcb-4ff1-af75-413cc162fb88 service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:47.959342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1353c3-3bcb-4ff1-af75-413cc162fb88 service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:47.959936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1353c3-3bcb-4ff1-af75-413cc162fb88 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:47.963237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8f1353c3-3bcb-4ff1-af75-413cc162fb88 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:47.997395 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1353c3-3bcb-4ff1-af75-413cc162fb88 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:47.997827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1353c3-3bcb-4ff1-af75-413cc162fb88 service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:48.010439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24228f81-6190-498e-aa79-0229587315ac service nova] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:48.011903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-24228f81-6190-498e-aa79-0229587315ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:48.018953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1208/2354] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:48.030582 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24228f81-6190-498e-aa79-0229587315ac service nova] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:48.035468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99925f0-19df-4341-bf13-331fbc5547e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:48.036339 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99925f0-19df-4341-bf13-331fbc5547e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:48.036629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99925f0-19df-4341-bf13-331fbc5547e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:48.036940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99925f0-19df-4341-bf13-331fbc5547e2 admin admin] PATH_INFO: `/v3/users/53f1188f2446493baae7dc79bbb66ff7` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:48.038162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99925f0-19df-4341-bf13-331fbc5547e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53f1188f2446493baae7dc79bbb66ff7)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:48.040094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99925f0-19df-4341-bf13-331fbc5547e2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:48.091767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c99925f0-19df-4341-bf13-331fbc5547e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53f1188f2446493baae7dc79bbb66ff7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:48.092881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c99925f0-19df-4341-bf13-331fbc5547e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53f1188f2446493baae7dc79bbb66ff7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:48.093527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c99925f0-19df-4341-bf13-331fbc5547e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53f1188f2446493baae7dc79bbb66ff7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:48.106379 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c99925f0-19df-4341-bf13-331fbc5547e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53f1188f2446493baae7dc79bbb66ff7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:48.113924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c99925f0-19df-4341-bf13-331fbc5547e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53f1188f2446493baae7dc79bbb66ff7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:48.128967 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1147/2355] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:00:48 2022] DELETE /identity/v3/users/53f1188f2446493baae7dc79bbb66ff7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:00:48.307656 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1353c3-3bcb-4ff1-af75-413cc162fb88 service placement] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:48.321643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1353c3-3bcb-4ff1-af75-413cc162fb88 service placement] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:48.406250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402fb8b5-616e-4bd9-b6ff-7821a66e219c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:48.407147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fb8b5-616e-4bd9-b6ff-7821a66e219c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:48.407438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fb8b5-616e-4bd9-b6ff-7821a66e219c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:48.408356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402fb8b5-616e-4bd9-b6ff-7821a66e219c admin admin] PATH_INFO: `/v3/projects/9353d04a04ab44e58f56fd01ca43d62e` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:48.409860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fb8b5-616e-4bd9-b6ff-7821a66e219c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9353d04a04ab44e58f56fd01ca43d62e)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:48.412964 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402fb8b5-616e-4bd9-b6ff-7821a66e219c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:48.482567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-402fb8b5-616e-4bd9-b6ff-7821a66e219c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9353d04a04ab44e58f56fd01ca43d62e', 'request_id': 'req-402fb8b5-616e-4bd9-b6ff-7821a66e219c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:48.498808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-402fb8b5-616e-4bd9-b6ff-7821a66e219c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9353d04a04ab44e58f56fd01ca43d62e', 'request_id': 'req-402fb8b5-616e-4bd9-b6ff-7821a66e219c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:48.510751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-402fb8b5-616e-4bd9-b6ff-7821a66e219c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9353d04a04ab44e58f56fd01ca43d62e', 'request_id': 'req-402fb8b5-616e-4bd9-b6ff-7821a66e219c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:48.524124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-402fb8b5-616e-4bd9-b6ff-7821a66e219c admin admin] The token cache is being invalidate because project 9353d04a04ab44e58f56fd01ca43d62e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:00:48.524766 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-402fb8b5-616e-4bd9-b6ff-7821a66e219c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:48.525655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-402fb8b5-616e-4bd9-b6ff-7821a66e219c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:48.530868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1209/2356] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:00:48 2022] DELETE /identity/v3/projects/9353d04a04ab44e58f56fd01ca43d62e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:48.532225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e21e8c-3e77-48a5-9695-d2088dafa41f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:48.532518 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e21e8c-3e77-48a5-9695-d2088dafa41f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:48.532854 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e21e8c-3e77-48a5-9695-d2088dafa41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:48.550833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-39e21e8c-3e77-48a5-9695-d2088dafa41f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:48.552069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-39e21e8c-3e77-48a5-9695-d2088dafa41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:48.613685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1148/2357] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:00:48.621251 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfee9a-84b4-4f92-bc5d-e89b5452fcbc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:48.621579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfee9a-84b4-4f92-bc5d-e89b5452fcbc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:48.622032 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bfee9a-84b4-4f92-bc5d-e89b5452fcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:48.641436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-55bfee9a-84b4-4f92-bc5d-e89b5452fcbc None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:48.642669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-55bfee9a-84b4-4f92-bc5d-e89b5452fcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:48.651124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1210/2358] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:00:48.660057 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e21e8c-3e77-48a5-9695-d2088dafa41f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:48.666404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e21e8c-3e77-48a5-9695-d2088dafa41f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:48.671808 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da534c7f-1575-4581-bf54-013f26fa556f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:48.672476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da534c7f-1575-4581-bf54-013f26fa556f admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:48.672794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da534c7f-1575-4581-bf54-013f26fa556f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:48.673082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da534c7f-1575-4581-bf54-013f26fa556f admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:48.674095 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da534c7f-1575-4581-bf54-013f26fa556f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:00:48.674402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da534c7f-1575-4581-bf54-013f26fa556f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:48.676182 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da534c7f-1575-4581-bf54-013f26fa556f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:48.689791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1149/2359] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:00:48.694573 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfee9a-84b4-4f92-bc5d-e89b5452fcbc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:48.700306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:48.704415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bfee9a-84b4-4f92-bc5d-e89b5452fcbc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:48.710480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447f2f8-5615-45ec-b96c-5b1eb566c454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:48.711187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447f2f8-5615-45ec-b96c-5b1eb566c454 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:48.711752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447f2f8-5615-45ec-b96c-5b1eb566c454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:48.712248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4447f2f8-5615-45ec-b96c-5b1eb566c454 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:48.713219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447f2f8-5615-45ec-b96c-5b1eb566c454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:48.715146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4447f2f8-5615-45ec-b96c-5b1eb566c454 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:48.743523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1211/2360] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:00:48 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:00:48.748400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da534c7f-1575-4581-bf54-013f26fa556f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:48.758849 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da534c7f-1575-4581-bf54-013f26fa556f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:48.807628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:48.822981 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c007-428c-4031-b24e-9f3b72719d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:48.823646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c007-428c-4031-b24e-9f3b72719d7f admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:48.823930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c007-428c-4031-b24e-9f3b72719d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:48.824206 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169c007-428c-4031-b24e-9f3b72719d7f admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:48.824919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169c007-428c-4031-b24e-9f3b72719d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:48.826823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169c007-428c-4031-b24e-9f3b72719d7f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:48.846366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1150/2361] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:00:48 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:00:48.850522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447f2f8-5615-45ec-b96c-5b1eb566c454 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:48.861942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4447f2f8-5615-45ec-b96c-5b1eb566c454 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:48.866728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5363d-5aa8-4b6a-96b4-bd3a78119964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:48.867326 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5363d-5aa8-4b6a-96b4-bd3a78119964 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:48.867615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5363d-5aa8-4b6a-96b4-bd3a78119964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:48.867896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5363d-5aa8-4b6a-96b4-bd3a78119964 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:48.868630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5363d-5aa8-4b6a-96b4-bd3a78119964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:48.870271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5363d-5aa8-4b6a-96b4-bd3a78119964 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:48.881965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1212/2362] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:00:48.886143 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c007-428c-4031-b24e-9f3b72719d7f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:48.896330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169c007-428c-4031-b24e-9f3b72719d7f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:48.901034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20274e0-1f58-453c-b656-e4bb8f87dead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:48.901689 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20274e0-1f58-453c-b656-e4bb8f87dead admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:48.901978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20274e0-1f58-453c-b656-e4bb8f87dead admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:48.902262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20274e0-1f58-453c-b656-e4bb8f87dead admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:48.902996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20274e0-1f58-453c-b656-e4bb8f87dead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:48.904630 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20274e0-1f58-453c-b656-e4bb8f87dead admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:48.914666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1151/2363] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:00:48.918777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5363d-5aa8-4b6a-96b4-bd3a78119964 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:48.929721 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5363d-5aa8-4b6a-96b4-bd3a78119964 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:48.934195 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e617c38-1136-4b63-9bed-f113991a0262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:48.935664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e617c38-1136-4b63-9bed-f113991a0262 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:48.936235 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e617c38-1136-4b63-9bed-f113991a0262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:48.936235 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e617c38-1136-4b63-9bed-f113991a0262 admin admin] PATH_INFO: `/v3/projects/fab58120fd3841d39020f59aafa4354c/users/2766137e0e334532a24681aa4385dd80/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:48.950390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e617c38-1136-4b63-9bed-f113991a0262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab58120fd3841d39020f59aafa4354c, user_id=2766137e0e334532a24681aa4385dd80, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:48.953053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e617c38-1136-4b63-9bed-f113991a0262 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:48.967536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1213/2364] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:00:48 2022] PUT /identity/v3/projects/fab58120fd3841d39020f59aafa4354c/users/2766137e0e334532a24681aa4385dd80/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:48.977614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f950909-fa98-4b86-8738-053e1fc7acef None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:48.977921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f950909-fa98-4b86-8738-053e1fc7acef None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:48.978209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f950909-fa98-4b86-8738-053e1fc7acef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:49.011367 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2f950909-fa98-4b86-8738-053e1fc7acef None None] MFA Rules not processed for user `2766137e0e334532a24681aa4385dd80`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:49.012575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2f950909-fa98-4b86-8738-053e1fc7acef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:49.076852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1152/2365] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:48 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:00:49.081300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e617c38-1136-4b63-9bed-f113991a0262 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:49.141653 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e617c38-1136-4b63-9bed-f113991a0262 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:49.146234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d186a-f3f0-4951-9c6a-f9b8c954e333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:49.146889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d186a-f3f0-4951-9c6a-f9b8c954e333 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:49.147178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d186a-f3f0-4951-9c6a-f9b8c954e333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:49.147456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d186a-f3f0-4951-9c6a-f9b8c954e333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:49.148153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d186a-f3f0-4951-9c6a-f9b8c954e333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:49.150106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d186a-f3f0-4951-9c6a-f9b8c954e333 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:49.177241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1214/2366] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:00:49 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:00:49.181741 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f950909-fa98-4b86-8738-053e1fc7acef None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:49.196576 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f950909-fa98-4b86-8738-053e1fc7acef None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:49.255834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9da68-b338-45d9-9830-7235ae604e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:49.256503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9da68-b338-45d9-9830-7235ae604e7b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:49.256820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9da68-b338-45d9-9830-7235ae604e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:49.257101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9da68-b338-45d9-9830-7235ae604e7b admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:49.258121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9da68-b338-45d9-9830-7235ae604e7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:49.259761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9da68-b338-45d9-9830-7235ae604e7b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:49.281663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1153/2367] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:00:49 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:00:49.285767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d186a-f3f0-4951-9c6a-f9b8c954e333 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:49.296887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d186a-f3f0-4951-9c6a-f9b8c954e333 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:49.302069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cced13-15aa-4724-b034-b3d114ad5364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:49.302754 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cced13-15aa-4724-b034-b3d114ad5364 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:49.303047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cced13-15aa-4724-b034-b3d114ad5364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:49.303353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cced13-15aa-4724-b034-b3d114ad5364 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:49.304095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cced13-15aa-4724-b034-b3d114ad5364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:49.305740 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cced13-15aa-4724-b034-b3d114ad5364 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:49.319854 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1215/2368] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:00:49.324098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9da68-b338-45d9-9830-7235ae604e7b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:49.335035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9da68-b338-45d9-9830-7235ae604e7b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:49.339862 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce872f-70d6-4efd-b8b7-27f1a548132e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:49.340798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce872f-70d6-4efd-b8b7-27f1a548132e admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:49.341127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce872f-70d6-4efd-b8b7-27f1a548132e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:49.341412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce872f-70d6-4efd-b8b7-27f1a548132e admin admin] PATH_INFO: `/v3/projects/6033403e0f3f49bf8882b2eb07bb01f1/users/bc8948869231430aaca534c6ea42f31c/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:49.358227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce872f-70d6-4efd-b8b7-27f1a548132e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6033403e0f3f49bf8882b2eb07bb01f1, user_id=bc8948869231430aaca534c6ea42f31c, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:49.360961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce872f-70d6-4efd-b8b7-27f1a548132e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:49.375817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1154/2369] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:00:49 2022] PUT /identity/v3/projects/6033403e0f3f49bf8882b2eb07bb01f1/users/bc8948869231430aaca534c6ea42f31c/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:49.379735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cced13-15aa-4724-b034-b3d114ad5364 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:49.390417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cced13-15aa-4724-b034-b3d114ad5364 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:49.453670 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c06bc7-5a94-4951-a6cc-e92611317139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:49.454259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c06bc7-5a94-4951-a6cc-e92611317139 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:49.454551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c06bc7-5a94-4951-a6cc-e92611317139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:49.454827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c06bc7-5a94-4951-a6cc-e92611317139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:49.455566 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c06bc7-5a94-4951-a6cc-e92611317139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:49.457217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c06bc7-5a94-4951-a6cc-e92611317139 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:49.471424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1216/2370] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:00:49.473628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce872f-70d6-4efd-b8b7-27f1a548132e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:49.485525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce872f-70d6-4efd-b8b7-27f1a548132e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:49.490661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47fac3-c1b8-485e-a526-a7d7605e4ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:49.491650 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47fac3-c1b8-485e-a526-a7d7605e4ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:49.492028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47fac3-c1b8-485e-a526-a7d7605e4ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:49.492379 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47fac3-c1b8-485e-a526-a7d7605e4ff4 admin admin] PATH_INFO: `/v3/projects/6033403e0f3f49bf8882b2eb07bb01f1/users/bc8948869231430aaca534c6ea42f31c/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:49.494820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47fac3-c1b8-485e-a526-a7d7605e4ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6033403e0f3f49bf8882b2eb07bb01f1, user_id=bc8948869231430aaca534c6ea42f31c, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:49.498258 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca47fac3-c1b8-485e-a526-a7d7605e4ff4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:49.510506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1155/2371] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:00:49 2022] PUT /identity/v3/projects/6033403e0f3f49bf8882b2eb07bb01f1/users/bc8948869231430aaca534c6ea42f31c/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:49.522110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f35943-7789-4684-8fd5-31a83b1d243c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:49.522430 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f35943-7789-4684-8fd5-31a83b1d243c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:49.522720 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f35943-7789-4684-8fd5-31a83b1d243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:49.553994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c3f35943-7789-4684-8fd5-31a83b1d243c None None] MFA Rules not processed for user `bc8948869231430aaca534c6ea42f31c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:49.555262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c3f35943-7789-4684-8fd5-31a83b1d243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:49.561548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:49.635229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1217/2372] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:00:49.647216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47fac3-c1b8-485e-a526-a7d7605e4ff4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:49.649151 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ca47fac3-c1b8-485e-a526-a7d7605e4ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:49.663264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:49.725990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca47fac3-c1b8-485e-a526-a7d7605e4ff4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:49.730031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4edb7-cc33-4691-a1b9-ee2407a2d97b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:49.730684 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4edb7-cc33-4691-a1b9-ee2407a2d97b service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:49.730976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4edb7-cc33-4691-a1b9-ee2407a2d97b service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:49.731255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4edb7-cc33-4691-a1b9-ee2407a2d97b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:49.747690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4edb7-cc33-4691-a1b9-ee2407a2d97b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:49.749216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4edb7-cc33-4691-a1b9-ee2407a2d97b service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:49.759852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1156/2373] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:49 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:50.138693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f35943-7789-4684-8fd5-31a83b1d243c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:50.152455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f35943-7789-4684-8fd5-31a83b1d243c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:50.158837 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239b842-8c90-478b-a517-72c6fef1306b tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc8948869231430aaca534c6ea42f31c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6033403e0f3f49bf8882b2eb07bb01f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:50.159987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239b842-8c90-478b-a517-72c6fef1306b tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:50.159987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239b842-8c90-478b-a517-72c6fef1306b tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:50.160378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5239b842-8c90-478b-a517-72c6fef1306b tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:50.160863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239b842-8c90-478b-a517-72c6fef1306b tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:00:50.161180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239b842-8c90-478b-a517-72c6fef1306b tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:50.163052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5239b842-8c90-478b-a517-72c6fef1306b tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:50.182715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1218/2374] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:00:50.184590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4edb7-cc33-4691-a1b9-ee2407a2d97b service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:50.195840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4edb7-cc33-4691-a1b9-ee2407a2d97b service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:50.201829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e3e37-f595-431f-b255-1842953ae108 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc8948869231430aaca534c6ea42f31c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6033403e0f3f49bf8882b2eb07bb01f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:50.202891 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e3e37-f595-431f-b255-1842953ae108 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:50.203262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e3e37-f595-431f-b255-1842953ae108 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:50.203586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e3e37-f595-431f-b255-1842953ae108 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:50.204482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e3e37-f595-431f-b255-1842953ae108 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:50.206837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e3e37-f595-431f-b255-1842953ae108 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:50.237688 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1157/2375] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:00:50 2022] POST /identity/v3/projects => generated 354 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:00:50.238706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239b842-8c90-478b-a517-72c6fef1306b tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:50.249774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5239b842-8c90-478b-a517-72c6fef1306b tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:50.306796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e89b7-ca8e-430a-93c8-1163bb726998 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc8948869231430aaca534c6ea42f31c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6033403e0f3f49bf8882b2eb07bb01f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:50.307464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e89b7-ca8e-430a-93c8-1163bb726998 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:50.307754 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e89b7-ca8e-430a-93c8-1163bb726998 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:50.308044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e89b7-ca8e-430a-93c8-1163bb726998 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:50.308808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e89b7-ca8e-430a-93c8-1163bb726998 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:00:50.309124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e89b7-ca8e-430a-93c8-1163bb726998 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:50.311293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e89b7-ca8e-430a-93c8-1163bb726998 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:50.326112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1219/2376] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:00:50.363004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e3e37-f595-431f-b255-1842953ae108 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:50.373649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e3e37-f595-431f-b255-1842953ae108 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:50.378475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc8948869231430aaca534c6ea42f31c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6033403e0f3f49bf8882b2eb07bb01f1', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:50.379374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:50.379801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:50.380089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] PATH_INFO: `/v3/projects/e8bd53ca92cc4e6a93df7309908b4105` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:50.381465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e8bd53ca92cc4e6a93df7309908b4105)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:50.384775 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:50.442553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8bd53ca92cc4e6a93df7309908b4105', 'request_id': 'req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:50.451909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8bd53ca92cc4e6a93df7309908b4105', 'request_id': 'req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:50.460644 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8bd53ca92cc4e6a93df7309908b4105', 'request_id': 'req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:50.483957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] The token cache is being invalidate because project e8bd53ca92cc4e6a93df7309908b4105 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:00:50.486439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:50.488193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-407a953b-da6a-401f-8ad3-2d2a0d75b1b9 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:50.492909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1158/2377] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:00:50 2022] DELETE /identity/v3/projects/e8bd53ca92cc4e6a93df7309908b4105 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:50.814382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e89b7-ca8e-430a-93c8-1163bb726998 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:50.816538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-eb5e89b7-ca8e-430a-93c8-1163bb726998 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:50.949485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e89b7-ca8e-430a-93c8-1163bb726998 tempest-VolumeQuotaClassesTest-364926560 tempest-VolumeQuotaClassesTest-364926560-project-admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:51.036441 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:51.079301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:51.081639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:51.081639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:51.081639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] PATH_INFO: `/v3/users/2766137e0e334532a24681aa4385dd80` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:51.082187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2766137e0e334532a24681aa4385dd80)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:51.084197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:51.143988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2766137e0e334532a24681aa4385dd80'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:51.145237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2766137e0e334532a24681aa4385dd80'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:51.146292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2766137e0e334532a24681aa4385dd80'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:51.162355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2766137e0e334532a24681aa4385dd80'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:51.171352 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2766137e0e334532a24681aa4385dd80'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:51.187928 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1220/2378] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:00:50 2022] DELETE /identity/v3/users/2766137e0e334532a24681aa4385dd80 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:51.192449 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e2700-16cd-437e-a494-f931689fce98 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:51.192971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e2700-16cd-437e-a494-f931689fce98 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:51.193074 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e2700-16cd-437e-a494-f931689fce98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:51.212065 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6e0e2700-16cd-437e-a494-f931689fce98 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:51.213350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6e0e2700-16cd-437e-a494-f931689fce98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:51.289847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1159/2379] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:00:51.329996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:51.332341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:51.407006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072b861-5681-48fb-b109-d96f7863ef3f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:51.410716 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069c13b-23b9-401b-bf7b-e28df6a0af06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:51.411380 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069c13b-23b9-401b-bf7b-e28df6a0af06 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:51.411672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069c13b-23b9-401b-bf7b-e28df6a0af06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:51.411956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069c13b-23b9-401b-bf7b-e28df6a0af06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:51.429735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069c13b-23b9-401b-bf7b-e28df6a0af06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:51.431212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9069c13b-23b9-401b-bf7b-e28df6a0af06 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:51.444438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1221/2380] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:52.131845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e2700-16cd-437e-a494-f931689fce98 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:52.133758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6e0e2700-16cd-437e-a494-f931689fce98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:52.150623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e2700-16cd-437e-a494-f931689fce98 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:52.157052 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:52.157932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:52.158217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:52.158498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] PATH_INFO: `/v3/projects/fab58120fd3841d39020f59aafa4354c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:52.160464 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab58120fd3841d39020f59aafa4354c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:52.162914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:52.170830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:52.235770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab58120fd3841d39020f59aafa4354c', 'request_id': 'req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:52.251025 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab58120fd3841d39020f59aafa4354c', 'request_id': 'req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:52.259862 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab58120fd3841d39020f59aafa4354c', 'request_id': 'req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:52.282216 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] The token cache is being invalidate because project fab58120fd3841d39020f59aafa4354c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:00:52.282512 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:52.283264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:52.285441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1160/2381] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:00:52 2022] DELETE /identity/v3/projects/fab58120fd3841d39020f59aafa4354c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:52.289464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069c13b-23b9-401b-bf7b-e28df6a0af06 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:52.300055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069c13b-23b9-401b-bf7b-e28df6a0af06 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:52.363628 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:52.376897 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5e325f-c365-4ba5-94b7-5a21b8e572c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:52.377877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e325f-c365-4ba5-94b7-5a21b8e572c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:52.378265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e325f-c365-4ba5-94b7-5a21b8e572c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:52.378625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5e325f-c365-4ba5-94b7-5a21b8e572c3 admin admin] PATH_INFO: `/v3/users/bc8948869231430aaca534c6ea42f31c` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:52.380290 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e325f-c365-4ba5-94b7-5a21b8e572c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc8948869231430aaca534c6ea42f31c)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:52.382771 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5e325f-c365-4ba5-94b7-5a21b8e572c3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:52.449933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7f5e325f-c365-4ba5-94b7-5a21b8e572c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc8948869231430aaca534c6ea42f31c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:52.450743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7f5e325f-c365-4ba5-94b7-5a21b8e572c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc8948869231430aaca534c6ea42f31c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:52.451418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7f5e325f-c365-4ba5-94b7-5a21b8e572c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc8948869231430aaca534c6ea42f31c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:52.477359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7f5e325f-c365-4ba5-94b7-5a21b8e572c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc8948869231430aaca534c6ea42f31c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:52.486388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7f5e325f-c365-4ba5-94b7-5a21b8e572c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc8948869231430aaca534c6ea42f31c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:52.510835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1222/2382] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:00:52 2022] DELETE /identity/v3/users/bc8948869231430aaca534c6ea42f31c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:00:52.981241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:52.982913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:53.000880 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e00f4-eb11-4a9a-8d20-04ea6086b1ff admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:53.063501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b35ee4-9bba-405a-b279-2655b95e984b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:53.064441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b35ee4-9bba-405a-b279-2655b95e984b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:53.064764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b35ee4-9bba-405a-b279-2655b95e984b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:53.065048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b35ee4-9bba-405a-b279-2655b95e984b admin admin] PATH_INFO: `/v3/projects/6033403e0f3f49bf8882b2eb07bb01f1` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:53.066289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b35ee4-9bba-405a-b279-2655b95e984b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6033403e0f3f49bf8882b2eb07bb01f1)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:53.068206 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b35ee4-9bba-405a-b279-2655b95e984b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:53.117087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a7b35ee4-9bba-405a-b279-2655b95e984b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6033403e0f3f49bf8882b2eb07bb01f1', 'request_id': 'req-a7b35ee4-9bba-405a-b279-2655b95e984b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:53.125016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a7b35ee4-9bba-405a-b279-2655b95e984b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6033403e0f3f49bf8882b2eb07bb01f1', 'request_id': 'req-a7b35ee4-9bba-405a-b279-2655b95e984b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:53.131642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a7b35ee4-9bba-405a-b279-2655b95e984b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6033403e0f3f49bf8882b2eb07bb01f1', 'request_id': 'req-a7b35ee4-9bba-405a-b279-2655b95e984b'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:53.142104 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a7b35ee4-9bba-405a-b279-2655b95e984b admin admin] The token cache is being invalidate because project 6033403e0f3f49bf8882b2eb07bb01f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:00:53.142351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a7b35ee4-9bba-405a-b279-2655b95e984b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:53.142968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-a7b35ee4-9bba-405a-b279-2655b95e984b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:53.144548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1161/2383] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:00:52 2022] DELETE /identity/v3/projects/6033403e0f3f49bf8882b2eb07bb01f1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:53.149583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116272a4-e3a8-4b88-a542-cce6216fdcec None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:53.149959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116272a4-e3a8-4b88-a542-cce6216fdcec None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:53.150244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116272a4-e3a8-4b88-a542-cce6216fdcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:53.168223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-116272a4-e3a8-4b88-a542-cce6216fdcec None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:53.170138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-116272a4-e3a8-4b88-a542-cce6216fdcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:53.236732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1223/2384] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:00:53.243752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0790db-0115-46ae-94ee-809cee96ce16 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:53.247878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0790db-0115-46ae-94ee-809cee96ce16 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:53.247878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0790db-0115-46ae-94ee-809cee96ce16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:53.268693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ca0790db-0115-46ae-94ee-809cee96ce16 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:53.270344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ca0790db-0115-46ae-94ee-809cee96ce16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:53.279102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1162/2385] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:00:53.283474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116272a4-e3a8-4b88-a542-cce6216fdcec None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:53.293981 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116272a4-e3a8-4b88-a542-cce6216fdcec None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:53.300234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda61ba5-a1a3-40dc-94dd-4253729453a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:53.300944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda61ba5-a1a3-40dc-94dd-4253729453a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:53.301237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda61ba5-a1a3-40dc-94dd-4253729453a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:53.301530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda61ba5-a1a3-40dc-94dd-4253729453a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:53.302403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda61ba5-a1a3-40dc-94dd-4253729453a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:00:53.302766 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda61ba5-a1a3-40dc-94dd-4253729453a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:53.304347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda61ba5-a1a3-40dc-94dd-4253729453a4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:53.324557 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1224/2386] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:00:53.325538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0790db-0115-46ae-94ee-809cee96ce16 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:53.347110 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0790db-0115-46ae-94ee-809cee96ce16 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:53.357736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d81009-eef1-44e0-8ca0-f170363a1d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:53.358425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d81009-eef1-44e0-8ca0-f170363a1d04 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:53.358714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d81009-eef1-44e0-8ca0-f170363a1d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:53.358993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d81009-eef1-44e0-8ca0-f170363a1d04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:53.359705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d81009-eef1-44e0-8ca0-f170363a1d04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:53.361661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d81009-eef1-44e0-8ca0-f170363a1d04 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:53.385911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:53.427886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1163/2387] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:00:53 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:00:53.429423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda61ba5-a1a3-40dc-94dd-4253729453a4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:53.442102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda61ba5-a1a3-40dc-94dd-4253729453a4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:53.511334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaa1bc-f8bb-43aa-af09-e8b6e083d405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:53.512172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaa1bc-f8bb-43aa-af09-e8b6e083d405 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:53.512591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaa1bc-f8bb-43aa-af09-e8b6e083d405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:53.513235 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaa1bc-f8bb-43aa-af09-e8b6e083d405 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:53.514053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaa1bc-f8bb-43aa-af09-e8b6e083d405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:53.516133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaa1bc-f8bb-43aa-af09-e8b6e083d405 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:53.525171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:53.554493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1225/2388] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:00:53 2022] POST /identity/v3/users => generated 296 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:00:53.556287 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d81009-eef1-44e0-8ca0-f170363a1d04 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:53.568833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d81009-eef1-44e0-8ca0-f170363a1d04 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:53.574703 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb377e-7c4e-4d6e-b5c7-360b0e8a8a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:53.575764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb377e-7c4e-4d6e-b5c7-360b0e8a8a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:53.576238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb377e-7c4e-4d6e-b5c7-360b0e8a8a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:53.576695 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb377e-7c4e-4d6e-b5c7-360b0e8a8a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:53.577694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb377e-7c4e-4d6e-b5c7-360b0e8a8a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:53.579525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb377e-7c4e-4d6e-b5c7-360b0e8a8a66 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:53.594127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1164/2389] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:00:53.595107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaa1bc-f8bb-43aa-af09-e8b6e083d405 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:53.608575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaa1bc-f8bb-43aa-af09-e8b6e083d405 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:53.616444 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009219b-1b28-42e3-b1c0-f066903dd30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:53.617296 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009219b-1b28-42e3-b1c0-f066903dd30c admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:53.617681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009219b-1b28-42e3-b1c0-f066903dd30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:53.618030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009219b-1b28-42e3-b1c0-f066903dd30c admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:53.618840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009219b-1b28-42e3-b1c0-f066903dd30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:53.620869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009219b-1b28-42e3-b1c0-f066903dd30c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:53.648628 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1226/2390] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:00:53.650487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb377e-7c4e-4d6e-b5c7-360b0e8a8a66 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:53.667064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb377e-7c4e-4d6e-b5c7-360b0e8a8a66 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:53.683442 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e09f1-f7ca-4e50-aa03-df7a6c361748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:53.685192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e09f1-f7ca-4e50-aa03-df7a6c361748 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:53.685661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e09f1-f7ca-4e50-aa03-df7a6c361748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:53.685944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763e09f1-f7ca-4e50-aa03-df7a6c361748 admin admin] PATH_INFO: `/v3/projects/674100995a964635ab2fc6a20b574485/users/753ea91309ae4269b5879df27fd9d3ee/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:53.704320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e09f1-f7ca-4e50-aa03-df7a6c361748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=674100995a964635ab2fc6a20b574485, user_id=753ea91309ae4269b5879df27fd9d3ee, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:53.707165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763e09f1-f7ca-4e50-aa03-df7a6c361748 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:53.723682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1165/2391] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:00:53 2022] PUT /identity/v3/projects/674100995a964635ab2fc6a20b574485/users/753ea91309ae4269b5879df27fd9d3ee/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:53.737617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d78adc-5478-4b83-a57c-de746755d461 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:53.737951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d78adc-5478-4b83-a57c-de746755d461 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:53.738207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d78adc-5478-4b83-a57c-de746755d461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:53.782523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-24d78adc-5478-4b83-a57c-de746755d461 None None] MFA Rules not processed for user `753ea91309ae4269b5879df27fd9d3ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:53.783914 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-24d78adc-5478-4b83-a57c-de746755d461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:53.854145 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1227/2392] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:00:53.858850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e09f1-f7ca-4e50-aa03-df7a6c361748 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:53.936399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763e09f1-f7ca-4e50-aa03-df7a6c361748 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:53.941844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adec9e7-8b63-4658-851b-a9474abb26fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:53.942516 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adec9e7-8b63-4658-851b-a9474abb26fd admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:53.942803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adec9e7-8b63-4658-851b-a9474abb26fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:53.943085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adec9e7-8b63-4658-851b-a9474abb26fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:53.943802 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adec9e7-8b63-4658-851b-a9474abb26fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:53.945787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adec9e7-8b63-4658-851b-a9474abb26fd admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:53.977204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1166/2393] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:00:53 2022] POST /identity/v3/projects => generated 391 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:00:53.978642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d78adc-5478-4b83-a57c-de746755d461 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:53.990173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d78adc-5478-4b83-a57c-de746755d461 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:54.059513 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77497b7-3e9c-4c71-898c-82276477f47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:54.060220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77497b7-3e9c-4c71-898c-82276477f47c admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:54.060507 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77497b7-3e9c-4c71-898c-82276477f47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:54.060868 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77497b7-3e9c-4c71-898c-82276477f47c admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:54.061575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77497b7-3e9c-4c71-898c-82276477f47c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:54.063558 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77497b7-3e9c-4c71-898c-82276477f47c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:54.087452 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1228/2394] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:00:53 2022] POST /identity/v3/users => generated 302 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:00:54.088629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adec9e7-8b63-4658-851b-a9474abb26fd admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:54.104038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adec9e7-8b63-4658-851b-a9474abb26fd admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:54.109076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb7e38-87d8-4d23-892c-591adbadeb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:54.109735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb7e38-87d8-4d23-892c-591adbadeb74 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:54.110024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb7e38-87d8-4d23-892c-591adbadeb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:54.110308 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb7e38-87d8-4d23-892c-591adbadeb74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:54.111041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb7e38-87d8-4d23-892c-591adbadeb74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:54.112687 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb7e38-87d8-4d23-892c-591adbadeb74 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:54.124646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1167/2395] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:00:54.128840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77497b7-3e9c-4c71-898c-82276477f47c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:54.139623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77497b7-3e9c-4c71-898c-82276477f47c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:54.144707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe098d-14ef-4b30-aba6-ee8be2530fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:54.145638 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe098d-14ef-4b30-aba6-ee8be2530fdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:54.145937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe098d-14ef-4b30-aba6-ee8be2530fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:54.146223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe098d-14ef-4b30-aba6-ee8be2530fdc admin admin] PATH_INFO: `/v3/projects/c5265d9ba0ea4aa8be983b3fe4a2ece5/users/b54b8a69935c4b8c81f0295cc9cd1e2c/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:54.162114 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe098d-14ef-4b30-aba6-ee8be2530fdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5265d9ba0ea4aa8be983b3fe4a2ece5, user_id=b54b8a69935c4b8c81f0295cc9cd1e2c, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:54.164799 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabe098d-14ef-4b30-aba6-ee8be2530fdc admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:54.177801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1229/2396] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:00:54 2022] PUT /identity/v3/projects/c5265d9ba0ea4aa8be983b3fe4a2ece5/users/b54b8a69935c4b8c81f0295cc9cd1e2c/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:00:54.181777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb7e38-87d8-4d23-892c-591adbadeb74 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:54.192469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb7e38-87d8-4d23-892c-591adbadeb74 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:54.248468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184cce2-8dd9-4328-9a90-46dd765e826b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:54.249263 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184cce2-8dd9-4328-9a90-46dd765e826b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:54.249451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184cce2-8dd9-4328-9a90-46dd765e826b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:54.249735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184cce2-8dd9-4328-9a90-46dd765e826b admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:54.250464 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184cce2-8dd9-4328-9a90-46dd765e826b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:54.252080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184cce2-8dd9-4328-9a90-46dd765e826b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:54.262214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1168/2397] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:00:54.267732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe098d-14ef-4b30-aba6-ee8be2530fdc admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:54.278259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe098d-14ef-4b30-aba6-ee8be2530fdc admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:54.282965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858ec96-371b-4eb2-9cc7-1e2bae129c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:54.283842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858ec96-371b-4eb2-9cc7-1e2bae129c2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:54.284233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858ec96-371b-4eb2-9cc7-1e2bae129c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:54.284515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858ec96-371b-4eb2-9cc7-1e2bae129c2d admin admin] PATH_INFO: `/v3/projects/c5265d9ba0ea4aa8be983b3fe4a2ece5/users/b54b8a69935c4b8c81f0295cc9cd1e2c/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:54.286667 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858ec96-371b-4eb2-9cc7-1e2bae129c2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5265d9ba0ea4aa8be983b3fe4a2ece5, user_id=b54b8a69935c4b8c81f0295cc9cd1e2c, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:54.289400 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858ec96-371b-4eb2-9cc7-1e2bae129c2d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:54.300337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1230/2398] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:00:54 2022] PUT /identity/v3/projects/c5265d9ba0ea4aa8be983b3fe4a2ece5/users/b54b8a69935c4b8c81f0295cc9cd1e2c/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:54.311605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb1eb3-1594-4f5d-8fc8-cc4711adfdb8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:54.311893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb1eb3-1594-4f5d-8fc8-cc4711adfdb8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:54.312174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb1eb3-1594-4f5d-8fc8-cc4711adfdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:54.340794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-35bb1eb3-1594-4f5d-8fc8-cc4711adfdb8 None None] MFA Rules not processed for user `b54b8a69935c4b8c81f0295cc9cd1e2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:54.342080 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-35bb1eb3-1594-4f5d-8fc8-cc4711adfdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:54.351613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:54.413736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1169/2399] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:54 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:00:54.424392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858ec96-371b-4eb2-9cc7-1e2bae129c2d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:54.426216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3858ec96-371b-4eb2-9cc7-1e2bae129c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:54.449870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:54.486286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858ec96-371b-4eb2-9cc7-1e2bae129c2d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:54.489961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965181ef-3bfc-469e-8b3e-fb5fb48952a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:54.490607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965181ef-3bfc-469e-8b3e-fb5fb48952a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:54.490895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965181ef-3bfc-469e-8b3e-fb5fb48952a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:54.491176 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965181ef-3bfc-469e-8b3e-fb5fb48952a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:54.500654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965181ef-3bfc-469e-8b3e-fb5fb48952a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:54.502406 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965181ef-3bfc-469e-8b3e-fb5fb48952a6 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:54.513802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1231/2400] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:54.557642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb1eb3-1594-4f5d-8fc8-cc4711adfdb8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:54.559038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-35bb1eb3-1594-4f5d-8fc8-cc4711adfdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:54.571701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb1eb3-1594-4f5d-8fc8-cc4711adfdb8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:54.576228 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fd8a63-0b20-4c88-9e1a-4f12dd5a0d76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:54.576995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd8a63-0b20-4c88-9e1a-4f12dd5a0d76 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:54.577287 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd8a63-0b20-4c88-9e1a-4f12dd5a0d76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:54.577568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fd8a63-0b20-4c88-9e1a-4f12dd5a0d76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:54.634058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd8a63-0b20-4c88-9e1a-4f12dd5a0d76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:54.635504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fd8a63-0b20-4c88-9e1a-4f12dd5a0d76 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:54.644538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1170/2401] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:57.906964 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965181ef-3bfc-469e-8b3e-fb5fb48952a6 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:57.972257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965181ef-3bfc-469e-8b3e-fb5fb48952a6 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:57.976804 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:57.977662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:57.977940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:57.978213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] PATH_INFO: `/v3/users/753ea91309ae4269b5879df27fd9d3ee` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:57.979393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] RBAC: Authorizing `identity:delete_user(user_id=753ea91309ae4269b5879df27fd9d3ee)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:57.981338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:58.035180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '753ea91309ae4269b5879df27fd9d3ee'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.036018 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '753ea91309ae4269b5879df27fd9d3ee'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.036652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '753ea91309ae4269b5879df27fd9d3ee'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.050640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '753ea91309ae4269b5879df27fd9d3ee'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.058632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '753ea91309ae4269b5879df27fd9d3ee'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.073528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1232/2402] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:00:57 2022] DELETE /identity/v3/users/753ea91309ae4269b5879df27fd9d3ee => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:58.078526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2c7e4-954c-47f8-a476-c0e450ca2fa8 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:58.079060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2c7e4-954c-47f8-a476-c0e450ca2fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:58.079369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2c7e4-954c-47f8-a476-c0e450ca2fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:58.097108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-21e2c7e4-954c-47f8-a476-c0e450ca2fa8 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:58.098474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-21e2c7e4-954c-47f8-a476-c0e450ca2fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:58.160992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1171/2403] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:00:58.170762 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:58.172198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:58.219069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:58.232238 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cda21-39f3-4b33-8adb-a47c94ed869b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:58.236425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791557e-67f1-4331-913d-590b4f87851d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:58.237112 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791557e-67f1-4331-913d-590b4f87851d service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:58.237401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791557e-67f1-4331-913d-590b4f87851d service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:58.237682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791557e-67f1-4331-913d-590b4f87851d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:58.247994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791557e-67f1-4331-913d-590b4f87851d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:58.249462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791557e-67f1-4331-913d-590b4f87851d service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:58.259221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1233/2404] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:00:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:00:58.532071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2c7e4-954c-47f8-a476-c0e450ca2fa8 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:58.533761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-21e2c7e4-954c-47f8-a476-c0e450ca2fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:58.546846 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2c7e4-954c-47f8-a476-c0e450ca2fa8 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:58.556874 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:58.557846 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:58.558210 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:58.558502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] PATH_INFO: `/v3/projects/674100995a964635ab2fc6a20b574485` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:58.561160 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=674100995a964635ab2fc6a20b574485)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:58.563564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:58.608979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:58.618970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '674100995a964635ab2fc6a20b574485', 'request_id': 'req-8ca94546-cfdd-433c-a733-cdb612033b09'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.630337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '674100995a964635ab2fc6a20b574485', 'request_id': 'req-8ca94546-cfdd-433c-a733-cdb612033b09'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.639213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '674100995a964635ab2fc6a20b574485', 'request_id': 'req-8ca94546-cfdd-433c-a733-cdb612033b09'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.651200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] The token cache is being invalidate because project 674100995a964635ab2fc6a20b574485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:00:58.651499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.652170 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.656845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1172/2405] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:00:58 2022] DELETE /identity/v3/projects/674100995a964635ab2fc6a20b574485 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:58.658120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791557e-67f1-4331-913d-590b4f87851d service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:58.673388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791557e-67f1-4331-913d-590b4f87851d service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:58.745115 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c19604-ac3f-45a1-b229-2e0eaf6f1919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:58.746076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c19604-ac3f-45a1-b229-2e0eaf6f1919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:58.746370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c19604-ac3f-45a1-b229-2e0eaf6f1919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:58.747228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c19604-ac3f-45a1-b229-2e0eaf6f1919 admin admin] PATH_INFO: `/v3/users/b54b8a69935c4b8c81f0295cc9cd1e2c` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:58.748713 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c19604-ac3f-45a1-b229-2e0eaf6f1919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b54b8a69935c4b8c81f0295cc9cd1e2c)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:58.751341 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c19604-ac3f-45a1-b229-2e0eaf6f1919 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:58.808990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-08c19604-ac3f-45a1-b229-2e0eaf6f1919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b54b8a69935c4b8c81f0295cc9cd1e2c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.810116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-08c19604-ac3f-45a1-b229-2e0eaf6f1919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b54b8a69935c4b8c81f0295cc9cd1e2c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.810912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-08c19604-ac3f-45a1-b229-2e0eaf6f1919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b54b8a69935c4b8c81f0295cc9cd1e2c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.826796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-08c19604-ac3f-45a1-b229-2e0eaf6f1919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b54b8a69935c4b8c81f0295cc9cd1e2c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.835438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-08c19604-ac3f-45a1-b229-2e0eaf6f1919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b54b8a69935c4b8c81f0295cc9cd1e2c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:58.852010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1234/2406] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:00:58 2022] DELETE /identity/v3/users/b54b8a69935c4b8c81f0295cc9cd1e2c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:00:59.154740 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:59.156266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:59.170634 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca94546-cfdd-433c-a733-cdb612033b09 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:59.236380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5c58b-f109-415b-9ddb-9c1800d25bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:59.237799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5c58b-f109-415b-9ddb-9c1800d25bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:59.237799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5c58b-f109-415b-9ddb-9c1800d25bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:59.238350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5c58b-f109-415b-9ddb-9c1800d25bce admin admin] PATH_INFO: `/v3/projects/c5265d9ba0ea4aa8be983b3fe4a2ece5` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:59.239785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5c58b-f109-415b-9ddb-9c1800d25bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5265d9ba0ea4aa8be983b3fe4a2ece5)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:59.241939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5c58b-f109-415b-9ddb-9c1800d25bce admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:59.321854 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-32f5c58b-f109-415b-9ddb-9c1800d25bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5265d9ba0ea4aa8be983b3fe4a2ece5', 'request_id': 'req-32f5c58b-f109-415b-9ddb-9c1800d25bce'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:59.331281 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-32f5c58b-f109-415b-9ddb-9c1800d25bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5265d9ba0ea4aa8be983b3fe4a2ece5', 'request_id': 'req-32f5c58b-f109-415b-9ddb-9c1800d25bce'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:59.339373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-32f5c58b-f109-415b-9ddb-9c1800d25bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5265d9ba0ea4aa8be983b3fe4a2ece5', 'request_id': 'req-32f5c58b-f109-415b-9ddb-9c1800d25bce'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:59.353396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-32f5c58b-f109-415b-9ddb-9c1800d25bce admin admin] The token cache is being invalidate because project c5265d9ba0ea4aa8be983b3fe4a2ece5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:00:59.353707 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-32f5c58b-f109-415b-9ddb-9c1800d25bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:59.354429 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-32f5c58b-f109-415b-9ddb-9c1800d25bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:00:59.356897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1173/2407] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:00:59 2022] DELETE /identity/v3/projects/c5265d9ba0ea4aa8be983b3fe4a2ece5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:59.362028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1be52c-4c11-4ac9-9d69-e4b76aef9a59 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:59.362544 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1be52c-4c11-4ac9-9d69-e4b76aef9a59 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:59.363115 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1be52c-4c11-4ac9-9d69-e4b76aef9a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:59.382553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-aa1be52c-4c11-4ac9-9d69-e4b76aef9a59 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:59.383828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-aa1be52c-4c11-4ac9-9d69-e4b76aef9a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:59.465576 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1235/2408] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:00:59.472874 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3fd26-c9a0-41cd-86e5-fe8af7629e9d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:59.473532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3fd26-c9a0-41cd-86e5-fe8af7629e9d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:59.473919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3fd26-c9a0-41cd-86e5-fe8af7629e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:59.495165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e4a3fd26-c9a0-41cd-86e5-fe8af7629e9d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:00:59.496868 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e4a3fd26-c9a0-41cd-86e5-fe8af7629e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:00:59.508315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1174/2409] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:00:59.512292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1be52c-4c11-4ac9-9d69-e4b76aef9a59 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:59.518913 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:59.523395 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1be52c-4c11-4ac9-9d69-e4b76aef9a59 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:59.530374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55430ecb-03b1-4a46-b063-841ab38b0352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:59.530994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55430ecb-03b1-4a46-b063-841ab38b0352 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:59.531275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55430ecb-03b1-4a46-b063-841ab38b0352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:59.531549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55430ecb-03b1-4a46-b063-841ab38b0352 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:59.532284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55430ecb-03b1-4a46-b063-841ab38b0352 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:00:59.532591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55430ecb-03b1-4a46-b063-841ab38b0352 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:59.534462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55430ecb-03b1-4a46-b063-841ab38b0352 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:59.549002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1236/2410] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:00:59.553051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3fd26-c9a0-41cd-86e5-fe8af7629e9d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:59.566929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3fd26-c9a0-41cd-86e5-fe8af7629e9d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:59.573447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5438bb6-0c3b-46be-9cb9-49bccbcb4422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:59.574127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5438bb6-0c3b-46be-9cb9-49bccbcb4422 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:59.574419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5438bb6-0c3b-46be-9cb9-49bccbcb4422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:59.575360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5438bb6-0c3b-46be-9cb9-49bccbcb4422 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:59.576738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5438bb6-0c3b-46be-9cb9-49bccbcb4422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:59.578698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5438bb6-0c3b-46be-9cb9-49bccbcb4422 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:59.612069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1175/2411] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:00:59 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:00:59.613621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55430ecb-03b1-4a46-b063-841ab38b0352 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:59.625543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55430ecb-03b1-4a46-b063-841ab38b0352 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:59.705721 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0298c6-6590-4f4e-801f-7ace127d716c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:59.706892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0298c6-6590-4f4e-801f-7ace127d716c admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:59.707370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0298c6-6590-4f4e-801f-7ace127d716c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:59.707944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0298c6-6590-4f4e-801f-7ace127d716c admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:59.709863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0298c6-6590-4f4e-801f-7ace127d716c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:59.712454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0298c6-6590-4f4e-801f-7ace127d716c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:59.742181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1237/2412] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:00:59 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:00:59.746460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5438bb6-0c3b-46be-9cb9-49bccbcb4422 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:59.760200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5438bb6-0c3b-46be-9cb9-49bccbcb4422 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:59.769593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d5100-dd83-4454-b8c0-02caa9606144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:59.770357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d5100-dd83-4454-b8c0-02caa9606144 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:59.770725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d5100-dd83-4454-b8c0-02caa9606144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:59.770992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d5100-dd83-4454-b8c0-02caa9606144 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:59.772014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d5100-dd83-4454-b8c0-02caa9606144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:59.773804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d5100-dd83-4454-b8c0-02caa9606144 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:59.791396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1176/2413] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:00:59.791860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0298c6-6590-4f4e-801f-7ace127d716c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:59.804255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0298c6-6590-4f4e-801f-7ace127d716c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:59.812872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742a089-0f2b-48b1-ae3b-b13914388054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:59.813565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742a089-0f2b-48b1-ae3b-b13914388054 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:59.813915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742a089-0f2b-48b1-ae3b-b13914388054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:59.814204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742a089-0f2b-48b1-ae3b-b13914388054 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:59.815127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742a089-0f2b-48b1-ae3b-b13914388054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:59.817672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742a089-0f2b-48b1-ae3b-b13914388054 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:59.833940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1238/2414] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:00:59.838232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d5100-dd83-4454-b8c0-02caa9606144 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:00:59.853110 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d5100-dd83-4454-b8c0-02caa9606144 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:00:59.860188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008d042-a2c2-418f-8f50-25ff42401577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:00:59.861601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008d042-a2c2-418f-8f50-25ff42401577 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:59.862010 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008d042-a2c2-418f-8f50-25ff42401577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:59.862398 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008d042-a2c2-418f-8f50-25ff42401577 admin admin] PATH_INFO: `/v3/projects/1ef9ed6f90754fdfb6a4cd3d6b7d8abb/users/860008e910bd4689bef3d02c61a27625/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:00:59.881622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008d042-a2c2-418f-8f50-25ff42401577 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef9ed6f90754fdfb6a4cd3d6b7d8abb, user_id=860008e910bd4689bef3d02c61a27625, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:00:59.884643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008d042-a2c2-418f-8f50-25ff42401577 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:00:59.888495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:00:59.933255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1177/2415] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:00:59 2022] PUT /identity/v3/projects/1ef9ed6f90754fdfb6a4cd3d6b7d8abb/users/860008e910bd4689bef3d02c61a27625/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:00:59.947230 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9869fd5-8e0b-4bb0-a2e1-25106a299c1f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:00:59.947706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9869fd5-8e0b-4bb0-a2e1-25106a299c1f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:00:59.948096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9869fd5-8e0b-4bb0-a2e1-25106a299c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:00.000074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b9869fd5-8e0b-4bb0-a2e1-25106a299c1f None None] MFA Rules not processed for user `860008e910bd4689bef3d02c61a27625`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:00.001432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b9869fd5-8e0b-4bb0-a2e1-25106a299c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:00.070047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1239/2416] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:00:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:00.074519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008d042-a2c2-418f-8f50-25ff42401577 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:00.143975 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008d042-a2c2-418f-8f50-25ff42401577 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:00.149085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03166f-8872-48fb-9b6b-0627e1d44c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:00.149823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03166f-8872-48fb-9b6b-0627e1d44c43 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:00.150180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03166f-8872-48fb-9b6b-0627e1d44c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:00.150535 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03166f-8872-48fb-9b6b-0627e1d44c43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:00.151311 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03166f-8872-48fb-9b6b-0627e1d44c43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:00.153378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d03166f-8872-48fb-9b6b-0627e1d44c43 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:00.180330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1178/2417] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:01:00 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:01:00.184274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9869fd5-8e0b-4bb0-a2e1-25106a299c1f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:00.197045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9869fd5-8e0b-4bb0-a2e1-25106a299c1f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:00.262296 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fffd7-8f69-4429-8146-6488ed2bb1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:00.263182 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fffd7-8f69-4429-8146-6488ed2bb1a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:00.263577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fffd7-8f69-4429-8146-6488ed2bb1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:00.264026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2fffd7-8f69-4429-8146-6488ed2bb1a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:00.264932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fffd7-8f69-4429-8146-6488ed2bb1a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:00.267039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2fffd7-8f69-4429-8146-6488ed2bb1a6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:00.274307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:00.290492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1240/2418] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:01:00 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:01:00.295043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03166f-8872-48fb-9b6b-0627e1d44c43 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:00.315903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03166f-8872-48fb-9b6b-0627e1d44c43 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:00.330352 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99afd7f-8216-4dff-961f-33fc9b467c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:00.330663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99afd7f-8216-4dff-961f-33fc9b467c6d admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:00.330957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99afd7f-8216-4dff-961f-33fc9b467c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:00.331240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99afd7f-8216-4dff-961f-33fc9b467c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:00.331983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99afd7f-8216-4dff-961f-33fc9b467c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:00.333624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99afd7f-8216-4dff-961f-33fc9b467c6d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:00.347861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1179/2419] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:01:00.352447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fffd7-8f69-4429-8146-6488ed2bb1a6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:00.365988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2fffd7-8f69-4429-8146-6488ed2bb1a6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:00.371818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b23fce-c655-41b4-a2ab-4c356ce752a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:00.374048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b23fce-c655-41b4-a2ab-4c356ce752a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:00.374443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b23fce-c655-41b4-a2ab-4c356ce752a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:00.374832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b23fce-c655-41b4-a2ab-4c356ce752a9 admin admin] PATH_INFO: `/v3/projects/3f78928f9c314423b588ace549ea57f5/users/c6e4f537db844c98aa2dae5178192881/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:00.395587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b23fce-c655-41b4-a2ab-4c356ce752a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f78928f9c314423b588ace549ea57f5, user_id=c6e4f537db844c98aa2dae5178192881, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:00.399022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b23fce-c655-41b4-a2ab-4c356ce752a9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:00.430833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1241/2420] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:01:00 2022] PUT /identity/v3/projects/3f78928f9c314423b588ace549ea57f5/users/c6e4f537db844c98aa2dae5178192881/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:00.435767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99afd7f-8216-4dff-961f-33fc9b467c6d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:00.452589 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99afd7f-8216-4dff-961f-33fc9b467c6d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:00.459889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62911d-fac5-426e-8871-ac6857b9df91 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:00.460090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62911d-fac5-426e-8871-ac6857b9df91 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:00.460517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62911d-fac5-426e-8871-ac6857b9df91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:00.482590 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7f62911d-fac5-426e-8871-ac6857b9df91 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:00.483832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7f62911d-fac5-426e-8871-ac6857b9df91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:00.536007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b26ee-dc39-4d48-b7ae-51bd4e800260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:00.536714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b26ee-dc39-4d48-b7ae-51bd4e800260 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:00.537365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b26ee-dc39-4d48-b7ae-51bd4e800260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:00.537651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b26ee-dc39-4d48-b7ae-51bd4e800260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:00.538812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b26ee-dc39-4d48-b7ae-51bd4e800260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:00.540725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b26ee-dc39-4d48-b7ae-51bd4e800260 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:00.556981 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1180/2421] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:01:00.557386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1242/2422] 10.222.0.67 () {58 vars in 967 bytes} [Thu Jun 23 12:01:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:00.558045 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b26ee-dc39-4d48-b7ae-51bd4e800260 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:00.570358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b26ee-dc39-4d48-b7ae-51bd4e800260 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:00.576410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a91a81-73a1-4153-8473-788bd2d1fd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:00.577409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a91a81-73a1-4153-8473-788bd2d1fd71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:00.577764 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a91a81-73a1-4153-8473-788bd2d1fd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:00.578111 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a91a81-73a1-4153-8473-788bd2d1fd71 admin admin] PATH_INFO: `/v3/projects/3f78928f9c314423b588ace549ea57f5/users/c6e4f537db844c98aa2dae5178192881/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:00.580203 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a91a81-73a1-4153-8473-788bd2d1fd71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f78928f9c314423b588ace549ea57f5, user_id=c6e4f537db844c98aa2dae5178192881, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:00.583217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a91a81-73a1-4153-8473-788bd2d1fd71 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:00.594980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1181/2423] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:01:00 2022] PUT /identity/v3/projects/3f78928f9c314423b588ace549ea57f5/users/c6e4f537db844c98aa2dae5178192881/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:01:00.605681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6242c-50c0-42e1-80e5-8a11af1d5af7 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:00.606034 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6242c-50c0-42e1-80e5-8a11af1d5af7 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:00.606247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6242c-50c0-42e1-80e5-8a11af1d5af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:00.607555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a91a81-73a1-4153-8473-788bd2d1fd71 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:00.609586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-25a91a81-73a1-4153-8473-788bd2d1fd71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:00.635017 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-80f6242c-50c0-42e1-80e5-8a11af1d5af7 None None] MFA Rules not processed for user `c6e4f537db844c98aa2dae5178192881`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:00.636285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-80f6242c-50c0-42e1-80e5-8a11af1d5af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:00.667952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:00.688320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a91a81-73a1-4153-8473-788bd2d1fd71 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:00.694257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b833d72-1638-429b-9ea7-756a135a3c35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:00.694914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b833d72-1638-429b-9ea7-756a135a3c35 service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:00.695245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b833d72-1638-429b-9ea7-756a135a3c35 service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:00.695474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b833d72-1638-429b-9ea7-756a135a3c35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:00.716895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1243/2424] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:00.727463 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6242c-50c0-42e1-80e5-8a11af1d5af7 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:00.729062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-80f6242c-50c0-42e1-80e5-8a11af1d5af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:00.761236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b833d72-1638-429b-9ea7-756a135a3c35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:00.762876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b833d72-1638-429b-9ea7-756a135a3c35 service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:00.772093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1182/2425] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 12:01:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 23 12:01:00.802641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6242c-50c0-42e1-80e5-8a11af1d5af7 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:00.806582 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665f7acd-30df-4719-95a8-826d58838af6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:00.807303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f7acd-30df-4719-95a8-826d58838af6 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:00.807677 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f7acd-30df-4719-95a8-826d58838af6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:00.808028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665f7acd-30df-4719-95a8-826d58838af6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:00.820197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f7acd-30df-4719-95a8-826d58838af6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:00.821713 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665f7acd-30df-4719-95a8-826d58838af6 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:00.826237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:00.834206 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1244/2426] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:01:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:01:01.542376 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b833d72-1638-429b-9ea7-756a135a3c35 service swift] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:01.613238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b833d72-1638-429b-9ea7-756a135a3c35 service swift] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:01.619629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:01.620835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:01.621316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:01.621802 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] PATH_INFO: `/v3/users/860008e910bd4689bef3d02c61a27625` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:01.624528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] RBAC: Authorizing `identity:delete_user(user_id=860008e910bd4689bef3d02c61a27625)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:01.626527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:01.695056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '860008e910bd4689bef3d02c61a27625'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:01.695813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '860008e910bd4689bef3d02c61a27625'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:01.696421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '860008e910bd4689bef3d02c61a27625'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:01.710624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '860008e910bd4689bef3d02c61a27625'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:01.718861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '860008e910bd4689bef3d02c61a27625'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:01.732903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1183/2427] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:01:01 2022] DELETE /identity/v3/users/860008e910bd4689bef3d02c61a27625 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:01.737780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f357f-f25a-443e-bdc5-5bb3c3ea7319 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:01.738288 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f357f-f25a-443e-bdc5-5bb3c3ea7319 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:01.738386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f357f-f25a-443e-bdc5-5bb3c3ea7319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:01.757458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-fb7f357f-f25a-443e-bdc5-5bb3c3ea7319 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:01.758703 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fb7f357f-f25a-443e-bdc5-5bb3c3ea7319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:01.838584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1245/2428] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:01:01.857280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:01.859403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:01.876378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:01.922889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a36f2-92f1-4b25-8270-aba6c73d403e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:01.927590 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d9571-0882-420d-9f03-2b6666a54d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:01.928060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d9571-0882-420d-9f03-2b6666a54d74 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:01.928560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d9571-0882-420d-9f03-2b6666a54d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:01.928884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545d9571-0882-420d-9f03-2b6666a54d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:01.942067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d9571-0882-420d-9f03-2b6666a54d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:01.943789 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545d9571-0882-420d-9f03-2b6666a54d74 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:01.956423 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1184/2429] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:01:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:01:02.357911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f357f-f25a-443e-bdc5-5bb3c3ea7319 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:02.360865 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fb7f357f-f25a-443e-bdc5-5bb3c3ea7319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:02.378598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f357f-f25a-443e-bdc5-5bb3c3ea7319 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:02.385169 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:02.389026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:02.389878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:02.389878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] PATH_INFO: `/v3/projects/1ef9ed6f90754fdfb6a4cd3d6b7d8abb` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:02.391706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ef9ed6f90754fdfb6a4cd3d6b7d8abb)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:02.393632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:02.478379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ef9ed6f90754fdfb6a4cd3d6b7d8abb', 'request_id': 'req-13a51007-8c5f-4eb9-859e-fc27e06a9c10'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:02.498730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ef9ed6f90754fdfb6a4cd3d6b7d8abb', 'request_id': 'req-13a51007-8c5f-4eb9-859e-fc27e06a9c10'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:02.532143 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ef9ed6f90754fdfb6a4cd3d6b7d8abb', 'request_id': 'req-13a51007-8c5f-4eb9-859e-fc27e06a9c10'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:02.547413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] The token cache is being invalidate because project 1ef9ed6f90754fdfb6a4cd3d6b7d8abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:01:02.547706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:02.548763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:02.553270 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1246/2430] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:01:02 2022] DELETE /identity/v3/projects/1ef9ed6f90754fdfb6a4cd3d6b7d8abb => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:02.554466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d9571-0882-420d-9f03-2b6666a54d74 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:02.564466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545d9571-0882-420d-9f03-2b6666a54d74 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:02.634612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa1ccb-0831-4852-8450-d51272ca1ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:02.636246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa1ccb-0831-4852-8450-d51272ca1ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:02.636773 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa1ccb-0831-4852-8450-d51272ca1ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:02.637197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa1ccb-0831-4852-8450-d51272ca1ce0 admin admin] PATH_INFO: `/v3/users/c6e4f537db844c98aa2dae5178192881` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:02.638668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa1ccb-0831-4852-8450-d51272ca1ce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6e4f537db844c98aa2dae5178192881)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:02.641626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa1ccb-0831-4852-8450-d51272ca1ce0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:02.704962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-acaa1ccb-0831-4852-8450-d51272ca1ce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6e4f537db844c98aa2dae5178192881'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:02.705755 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-acaa1ccb-0831-4852-8450-d51272ca1ce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6e4f537db844c98aa2dae5178192881'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:02.706375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-acaa1ccb-0831-4852-8450-d51272ca1ce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6e4f537db844c98aa2dae5178192881'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:02.723163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-acaa1ccb-0831-4852-8450-d51272ca1ce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6e4f537db844c98aa2dae5178192881'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:02.731588 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-acaa1ccb-0831-4852-8450-d51272ca1ce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6e4f537db844c98aa2dae5178192881'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:02.749897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1185/2431] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:01:02 2022] DELETE /identity/v3/users/c6e4f537db844c98aa2dae5178192881 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:03.337881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:03.339514 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:03.355937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a51007-8c5f-4eb9-859e-fc27e06a9c10 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:03.453705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964d01c-8f66-4969-a602-5c04bd7a8fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:03.454650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964d01c-8f66-4969-a602-5c04bd7a8fb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:03.454932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964d01c-8f66-4969-a602-5c04bd7a8fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:03.456256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964d01c-8f66-4969-a602-5c04bd7a8fb8 admin admin] PATH_INFO: `/v3/projects/3f78928f9c314423b588ace549ea57f5` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:03.457612 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964d01c-8f66-4969-a602-5c04bd7a8fb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f78928f9c314423b588ace549ea57f5)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:03.459833 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964d01c-8f66-4969-a602-5c04bd7a8fb8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:03.533124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:03.573414 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a964d01c-8f66-4969-a602-5c04bd7a8fb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f78928f9c314423b588ace549ea57f5', 'request_id': 'req-a964d01c-8f66-4969-a602-5c04bd7a8fb8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:03.588778 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a964d01c-8f66-4969-a602-5c04bd7a8fb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f78928f9c314423b588ace549ea57f5', 'request_id': 'req-a964d01c-8f66-4969-a602-5c04bd7a8fb8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:03.597346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a964d01c-8f66-4969-a602-5c04bd7a8fb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f78928f9c314423b588ace549ea57f5', 'request_id': 'req-a964d01c-8f66-4969-a602-5c04bd7a8fb8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:03.620308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a964d01c-8f66-4969-a602-5c04bd7a8fb8 admin admin] The token cache is being invalidate because project 3f78928f9c314423b588ace549ea57f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:01:03.620603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a964d01c-8f66-4969-a602-5c04bd7a8fb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:03.621579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-a964d01c-8f66-4969-a602-5c04bd7a8fb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:03.624022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1247/2432] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:01:03 2022] DELETE /identity/v3/projects/3f78928f9c314423b588ace549ea57f5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:03.628960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1c6c7-3bd3-4b39-9e1d-217452431e09 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:03.629453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1c6c7-3bd3-4b39-9e1d-217452431e09 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:03.629560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1c6c7-3bd3-4b39-9e1d-217452431e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:03.651002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b1f1c6c7-3bd3-4b39-9e1d-217452431e09 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:03.652585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b1f1c6c7-3bd3-4b39-9e1d-217452431e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:03.688968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:03.735544 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1186/2433] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:03.743743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9a5e2-da1e-4649-9eb3-9557a08d68c4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:03.744043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9a5e2-da1e-4649-9eb3-9557a08d68c4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:03.744336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9a5e2-da1e-4649-9eb3-9557a08d68c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:03.764116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bfd9a5e2-da1e-4649-9eb3-9557a08d68c4 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:03.765385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bfd9a5e2-da1e-4649-9eb3-9557a08d68c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:03.774487 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1248/2434] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:03.778791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1c6c7-3bd3-4b39-9e1d-217452431e09 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:03.788925 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1c6c7-3bd3-4b39-9e1d-217452431e09 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:03.795091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83edfa-aa9a-4d16-b906-b2d0bf304a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:03.795759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83edfa-aa9a-4d16-b906-b2d0bf304a96 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:03.796049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83edfa-aa9a-4d16-b906-b2d0bf304a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:03.796335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83edfa-aa9a-4d16-b906-b2d0bf304a96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:03.797103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83edfa-aa9a-4d16-b906-b2d0bf304a96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:01:03.797421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83edfa-aa9a-4d16-b906-b2d0bf304a96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:03.799300 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83edfa-aa9a-4d16-b906-b2d0bf304a96 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:03.813332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1187/2435] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:01:03.817710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9a5e2-da1e-4649-9eb3-9557a08d68c4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:03.827489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd9a5e2-da1e-4649-9eb3-9557a08d68c4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:03.833416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71da7a3-9c7d-40fe-9970-b60d334802d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:03.834711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71da7a3-9c7d-40fe-9970-b60d334802d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:03.835061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71da7a3-9c7d-40fe-9970-b60d334802d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:03.835346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71da7a3-9c7d-40fe-9970-b60d334802d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:03.836042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71da7a3-9c7d-40fe-9970-b60d334802d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:03.838107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71da7a3-9c7d-40fe-9970-b60d334802d4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:03.869035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1249/2436] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:01:03 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:01:03.874154 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83edfa-aa9a-4d16-b906-b2d0bf304a96 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:03.888867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83edfa-aa9a-4d16-b906-b2d0bf304a96 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:03.949863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52856c86-f344-4650-9542-bba50ae5ef5b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:03.950223 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52856c86-f344-4650-9542-bba50ae5ef5b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:03.950608 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52856c86-f344-4650-9542-bba50ae5ef5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:03.983710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-52856c86-f344-4650-9542-bba50ae5ef5b None None] MFA Rules not processed for user `9ab84989dd384d15b316b20cc65e792a`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:03.985347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-52856c86-f344-4650-9542-bba50ae5ef5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:03.990503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d3513-69a4-4e0f-8b85-c629804f900b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:03.991665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d3513-69a4-4e0f-8b85-c629804f900b admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:03.992009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d3513-69a4-4e0f-8b85-c629804f900b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:03.992734 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d3513-69a4-4e0f-8b85-c629804f900b admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:03.996219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d3513-69a4-4e0f-8b85-c629804f900b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:03.998771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d3513-69a4-4e0f-8b85-c629804f900b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:04.033362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1188/2437] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:01:03 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:01:04.037509 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d3513-69a4-4e0f-8b85-c629804f900b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:04.058990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d3513-69a4-4e0f-8b85-c629804f900b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:04.070010 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb52a6-4884-4587-83e9-00c3175e20e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:04.070690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb52a6-4884-4587-83e9-00c3175e20e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:04.070982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb52a6-4884-4587-83e9-00c3175e20e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:04.071262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb52a6-4884-4587-83e9-00c3175e20e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:04.072006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb52a6-4884-4587-83e9-00c3175e20e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:04.073662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb52a6-4884-4587-83e9-00c3175e20e9 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:04.084174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1250/2438] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:01:04.088143 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1189/2439] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:01:04.093396 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52856c86-f344-4650-9542-bba50ae5ef5b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:04.111787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52856c86-f344-4650-9542-bba50ae5ef5b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:04.129003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e90ac-5de5-4ac9-b4c0-5a2a5171ebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:04.129830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e90ac-5de5-4ac9-b4c0-5a2a5171ebd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:04.130201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e90ac-5de5-4ac9-b4c0-5a2a5171ebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:04.130556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e90ac-5de5-4ac9-b4c0-5a2a5171ebd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:04.131639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e90ac-5de5-4ac9-b4c0-5a2a5171ebd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:04.134124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5e90ac-5de5-4ac9-b4c0-5a2a5171ebd1 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:04.154932 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1251/2440] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:01:04.155923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb52a6-4884-4587-83e9-00c3175e20e9 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:04.167819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb52a6-4884-4587-83e9-00c3175e20e9 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:04.174602 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4311c-8b19-475f-9d0e-38a497a3b66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:04.175440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4311c-8b19-475f-9d0e-38a497a3b66a admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:04.175755 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4311c-8b19-475f-9d0e-38a497a3b66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:04.176036 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4311c-8b19-475f-9d0e-38a497a3b66a admin admin] PATH_INFO: `/v3/projects/c585cd4b502b4045ad389e3bfb0accee/users/f0d4301fec0c4bd1a10cb90e596cc379/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:04.194386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4311c-8b19-475f-9d0e-38a497a3b66a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c585cd4b502b4045ad389e3bfb0accee, user_id=f0d4301fec0c4bd1a10cb90e596cc379, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:04.197607 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4311c-8b19-475f-9d0e-38a497a3b66a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:04.222688 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1190/2441] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:01:04 2022] PUT /identity/v3/projects/c585cd4b502b4045ad389e3bfb0accee/users/f0d4301fec0c4bd1a10cb90e596cc379/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:01:04.234404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24c206-ff4b-40cc-8210-ea596a62c91c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:04.234705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24c206-ff4b-40cc-8210-ea596a62c91c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:04.235001 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24c206-ff4b-40cc-8210-ea596a62c91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:04.270134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5c24c206-ff4b-40cc-8210-ea596a62c91c None None] MFA Rules not processed for user `f0d4301fec0c4bd1a10cb90e596cc379`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:04.271366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5c24c206-ff4b-40cc-8210-ea596a62c91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:04.336841 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1252/2442] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:04.348249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4311c-8b19-475f-9d0e-38a497a3b66a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:04.349963 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d2c4311c-8b19-475f-9d0e-38a497a3b66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:04.407614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4311c-8b19-475f-9d0e-38a497a3b66a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:04.413554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a6454-8e0b-49b3-9b44-d71e085f6e7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74fdc12d728d45018ea668a4073da3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:04.414213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a6454-8e0b-49b3-9b44-d71e085f6e7a service glance] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:04.416631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a6454-8e0b-49b3-9b44-d71e085f6e7a service glance] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:04.416950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a6454-8e0b-49b3-9b44-d71e085f6e7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:04.429452 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a6454-8e0b-49b3-9b44-d71e085f6e7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:04.430855 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a6454-8e0b-49b3-9b44-d71e085f6e7a service glance] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:04.452882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1191/2443] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:01:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:01:09.482177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24c206-ff4b-40cc-8210-ea596a62c91c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:09.483915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5c24c206-ff4b-40cc-8210-ea596a62c91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:09.496077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:09.559058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24c206-ff4b-40cc-8210-ea596a62c91c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:09.564102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:09.565012 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:09.565409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:09.565701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] PATH_INFO: `/v3/users/2658206b0a934bd8861a49701ab41659` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:09.566908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2658206b0a934bd8861a49701ab41659)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:09.568866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:09.615735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2658206b0a934bd8861a49701ab41659'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:09.616317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2658206b0a934bd8861a49701ab41659'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:09.616956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2658206b0a934bd8861a49701ab41659'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:09.629212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2658206b0a934bd8861a49701ab41659'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:09.636232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2658206b0a934bd8861a49701ab41659'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:09.652106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1253/2444] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:01:09 2022] DELETE /identity/v3/users/2658206b0a934bd8861a49701ab41659 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:09.828552 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a6454-8e0b-49b3-9b44-d71e085f6e7a service glance] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:09.841690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a6454-8e0b-49b3-9b44-d71e085f6e7a service glance] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:09.865562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:09.906180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:09.907030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:09.907315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:09.907593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] PATH_INFO: `/v3/projects/45aa73614bc447bdb1bfda38ad541ab3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:09.908804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45aa73614bc447bdb1bfda38ad541ab3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:09.910709 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:09.958240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45aa73614bc447bdb1bfda38ad541ab3', 'request_id': 'req-3c90182f-9689-4969-a423-0653d7723444'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:09.965156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45aa73614bc447bdb1bfda38ad541ab3', 'request_id': 'req-3c90182f-9689-4969-a423-0653d7723444'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:09.971616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45aa73614bc447bdb1bfda38ad541ab3', 'request_id': 'req-3c90182f-9689-4969-a423-0653d7723444'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:09.983299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] The token cache is being invalidate because project 45aa73614bc447bdb1bfda38ad541ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:01:09.983589 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:09.986961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:09.988752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1192/2445] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:01:09 2022] DELETE /identity/v3/projects/45aa73614bc447bdb1bfda38ad541ab3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:01:09.993005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:09.994406 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:10.008113 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48805448-a0e0-4b03-9b47-a43bc6ec5536 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:10.076945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:10.077998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:10.078285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:10.078564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] PATH_INFO: `/v3/users/d990402118ea4227aa0f8feda9e42e61` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:10.080413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d990402118ea4227aa0f8feda9e42e61)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:10.082414 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:10.119468 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:10.144511 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd990402118ea4227aa0f8feda9e42e61'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:10.145409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd990402118ea4227aa0f8feda9e42e61'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:10.146227 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd990402118ea4227aa0f8feda9e42e61'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:10.159583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd990402118ea4227aa0f8feda9e42e61'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:10.169706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd990402118ea4227aa0f8feda9e42e61'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:10.183999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1254/2446] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:01:09 2022] DELETE /identity/v3/users/d990402118ea4227aa0f8feda9e42e61 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:10.450563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:10.452965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:10.497510 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c90182f-9689-4969-a423-0653d7723444 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:10.610324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:10.611202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:10.617665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:10.617665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb admin admin] PATH_INFO: `/v3/projects/1c4334d0e4a245f0a0a1ab8d6e35a89a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:10.618751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c4334d0e4a245f0a0a1ab8d6e35a89a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:10.622492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:10.700097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c4334d0e4a245f0a0a1ab8d6e35a89a', 'request_id': 'req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:10.709261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:10.718582 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c4334d0e4a245f0a0a1ab8d6e35a89a', 'request_id': 'req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:10.735261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c4334d0e4a245f0a0a1ab8d6e35a89a', 'request_id': 'req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:10.747232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb admin admin] The token cache is being invalidate because project 1c4334d0e4a245f0a0a1ab8d6e35a89a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:01:10.747540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:10.749081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5326afff-065a-47f8-9ccf-0cb32e5f0eeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:10.750132 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1193/2447] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:01:10 2022] DELETE /identity/v3/projects/1c4334d0e4a245f0a0a1ab8d6e35a89a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:20.183338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=19,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:20.743331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=3,keystone:SELECT=6 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:21.280374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:21.282505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:21.296044 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bee28f-e6a8-4c90-b2ea-d9f006c0101a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:21.357848 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:21.358774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:21.359057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:21.359333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] PATH_INFO: `/v3/users/f0d4301fec0c4bd1a10cb90e596cc379` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:21.360706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0d4301fec0c4bd1a10cb90e596cc379)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:21.362711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:21.410760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0d4301fec0c4bd1a10cb90e596cc379'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:21.411454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0d4301fec0c4bd1a10cb90e596cc379'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:21.412081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0d4301fec0c4bd1a10cb90e596cc379'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:21.423713 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0d4301fec0c4bd1a10cb90e596cc379'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:21.431006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0d4301fec0c4bd1a10cb90e596cc379'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:21.444370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1255/2448] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:01:21 2022] DELETE /identity/v3/users/f0d4301fec0c4bd1a10cb90e596cc379 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:21.449209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5124a-4b92-4caf-96ce-c7cab8e2c0ec None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:21.449460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5124a-4b92-4caf-96ce-c7cab8e2c0ec None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:21.449862 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d5124a-4b92-4caf-96ce-c7cab8e2c0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:21.470262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-20d5124a-4b92-4caf-96ce-c7cab8e2c0ec None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:21.471317 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-20d5124a-4b92-4caf-96ce-c7cab8e2c0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:21.545224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1194/2449] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:21.555362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:21.558270 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:21.617006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce26729-5c55-4e51-85a0-1ebb1048f1e0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:21.620688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff01a78-a9a2-4eb3-af34-0c4e8cc5d623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:21.621345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff01a78-a9a2-4eb3-af34-0c4e8cc5d623 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:21.621637 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff01a78-a9a2-4eb3-af34-0c4e8cc5d623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:21.621919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff01a78-a9a2-4eb3-af34-0c4e8cc5d623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:21.631483 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff01a78-a9a2-4eb3-af34-0c4e8cc5d623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:21.632934 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff01a78-a9a2-4eb3-af34-0c4e8cc5d623 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:21.642897 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1256/2450] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:01:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:01:21.911436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5124a-4b92-4caf-96ce-c7cab8e2c0ec None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:21.912952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-20d5124a-4b92-4caf-96ce-c7cab8e2c0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:21.928463 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d5124a-4b92-4caf-96ce-c7cab8e2c0ec None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:21.933564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:21.934445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:21.934730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:21.935009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c admin admin] PATH_INFO: `/v3/projects/c585cd4b502b4045ad389e3bfb0accee` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:21.936747 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c585cd4b502b4045ad389e3bfb0accee)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:21.939356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:21.994820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c585cd4b502b4045ad389e3bfb0accee', 'request_id': 'req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:22.002918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c585cd4b502b4045ad389e3bfb0accee', 'request_id': 'req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:22.009653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c585cd4b502b4045ad389e3bfb0accee', 'request_id': 'req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:22.020777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c admin admin] The token cache is being invalidate because project c585cd4b502b4045ad389e3bfb0accee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:01:22.021072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:22.021732 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-d284e70c-1c7e-49b0-b71c-3be8ffd5c77c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:22.023279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1195/2451] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:01:21 2022] DELETE /identity/v3/projects/c585cd4b502b4045ad389e3bfb0accee => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:22.028111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e708fe-8a69-470e-ad3f-fcd2369ff5f8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:22.028479 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e708fe-8a69-470e-ad3f-fcd2369ff5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:22.028940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e708fe-8a69-470e-ad3f-fcd2369ff5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:22.048638 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-29e708fe-8a69-470e-ad3f-fcd2369ff5f8 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:22.050134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-29e708fe-8a69-470e-ad3f-fcd2369ff5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:22.068221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:22.112034 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1257/2452] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:22.120149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685c7bf-ff2b-4862-b2c2-3be4aff29949 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:22.120439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685c7bf-ff2b-4862-b2c2-3be4aff29949 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:22.120754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f685c7bf-ff2b-4862-b2c2-3be4aff29949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:22.139344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-f685c7bf-ff2b-4862-b2c2-3be4aff29949 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:22.140540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f685c7bf-ff2b-4862-b2c2-3be4aff29949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:22.149441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1196/2453] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:22.153606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e708fe-8a69-470e-ad3f-fcd2369ff5f8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:22.163291 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e708fe-8a69-470e-ad3f-fcd2369ff5f8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:22.167915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8dad4-7abc-48ba-bb60-8a7a138f664e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:22.168575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8dad4-7abc-48ba-bb60-8a7a138f664e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:22.168892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8dad4-7abc-48ba-bb60-8a7a138f664e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:22.169172 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8dad4-7abc-48ba-bb60-8a7a138f664e admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:22.169950 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8dad4-7abc-48ba-bb60-8a7a138f664e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:01:22.170260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8dad4-7abc-48ba-bb60-8a7a138f664e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:22.172405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8dad4-7abc-48ba-bb60-8a7a138f664e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:22.184629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1258/2454] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:01:22.189666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685c7bf-ff2b-4862-b2c2-3be4aff29949 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:22.203911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f685c7bf-ff2b-4862-b2c2-3be4aff29949 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:22.208955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b3b1-fa9b-41b5-b829-3d0ffd4ce5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:22.209616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b3b1-fa9b-41b5-b829-3d0ffd4ce5bb admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:22.209903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b3b1-fa9b-41b5-b829-3d0ffd4ce5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:22.210180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b3b1-fa9b-41b5-b829-3d0ffd4ce5bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:22.210873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0b3b1-fa9b-41b5-b829-3d0ffd4ce5bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:22.212826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0b3b1-fa9b-41b5-b829-3d0ffd4ce5bb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:22.244357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1197/2455] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:01:22 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:01:22.244569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8dad4-7abc-48ba-bb60-8a7a138f664e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:22.255719 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8dad4-7abc-48ba-bb60-8a7a138f664e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:22.315997 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deac9c-3686-46f5-b375-cc488091203f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:22.316684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deac9c-3686-46f5-b375-cc488091203f admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:22.316908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deac9c-3686-46f5-b375-cc488091203f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:22.317194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deac9c-3686-46f5-b375-cc488091203f admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:22.317884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deac9c-3686-46f5-b375-cc488091203f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:22.319927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10deac9c-3686-46f5-b375-cc488091203f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:22.339050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1259/2456] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:01:22 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:01:22.343787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b3b1-fa9b-41b5-b829-3d0ffd4ce5bb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:22.354297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b3b1-fa9b-41b5-b829-3d0ffd4ce5bb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:22.359670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473207f-c96d-4e66-91de-c6a2e9754acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:22.360320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473207f-c96d-4e66-91de-c6a2e9754acb admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:22.360610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473207f-c96d-4e66-91de-c6a2e9754acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:22.360922 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473207f-c96d-4e66-91de-c6a2e9754acb admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:22.361643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473207f-c96d-4e66-91de-c6a2e9754acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:22.363246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473207f-c96d-4e66-91de-c6a2e9754acb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:22.374362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1198/2457] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:01:22.378263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deac9c-3686-46f5-b375-cc488091203f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:22.388231 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deac9c-3686-46f5-b375-cc488091203f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:22.392843 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f06a7-0a81-4704-b306-29cca8a3e2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:22.393495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f06a7-0a81-4704-b306-29cca8a3e2aa admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:22.393782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f06a7-0a81-4704-b306-29cca8a3e2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:22.394060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f06a7-0a81-4704-b306-29cca8a3e2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:22.394777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f06a7-0a81-4704-b306-29cca8a3e2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:22.396367 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1f06a7-0a81-4704-b306-29cca8a3e2aa admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:22.406791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1260/2458] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:01:22.410868 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473207f-c96d-4e66-91de-c6a2e9754acb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:22.421687 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473207f-c96d-4e66-91de-c6a2e9754acb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:22.426577 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e122dd-e947-4617-a9e8-0c640f0e385f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:22.427434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e122dd-e947-4617-a9e8-0c640f0e385f admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:22.427721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e122dd-e947-4617-a9e8-0c640f0e385f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:22.427994 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e122dd-e947-4617-a9e8-0c640f0e385f admin admin] PATH_INFO: `/v3/projects/7c97d4bfb04f4f79b662ce7e85d48a4b/users/dc5a97d65776461f8892b4575754ed12/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:22.443188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e122dd-e947-4617-a9e8-0c640f0e385f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c97d4bfb04f4f79b662ce7e85d48a4b, user_id=dc5a97d65776461f8892b4575754ed12, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:22.445945 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e122dd-e947-4617-a9e8-0c640f0e385f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:22.459064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1199/2459] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:01:22 2022] PUT /identity/v3/projects/7c97d4bfb04f4f79b662ce7e85d48a4b/users/dc5a97d65776461f8892b4575754ed12/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:22.471153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4658204-d273-4fea-b388-b2210756592d None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:22.471424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4658204-d273-4fea-b388-b2210756592d None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:22.471706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4658204-d273-4fea-b388-b2210756592d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:22.498895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c4658204-d273-4fea-b388-b2210756592d None None] MFA Rules not processed for user `dc5a97d65776461f8892b4575754ed12`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:22.500185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c4658204-d273-4fea-b388-b2210756592d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:22.570749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1261/2460] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:22.581246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e122dd-e947-4617-a9e8-0c640f0e385f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:22.583097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-30e122dd-e947-4617-a9e8-0c640f0e385f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:22.606041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:22.646658 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e122dd-e947-4617-a9e8-0c640f0e385f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:22.650519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951299be-5480-42b9-ac35-31c856743d33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:22.651156 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951299be-5480-42b9-ac35-31c856743d33 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:22.651445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951299be-5480-42b9-ac35-31c856743d33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:22.651719 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951299be-5480-42b9-ac35-31c856743d33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:22.661246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951299be-5480-42b9-ac35-31c856743d33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:22.663136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951299be-5480-42b9-ac35-31c856743d33 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:22.673422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1200/2461] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:01:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:01:30.497832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9830bf-a4c4-4f9b-8eb3-bde18580ca8a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:30.498318 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9830bf-a4c4-4f9b-8eb3-bde18580ca8a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:30.498432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9830bf-a4c4-4f9b-8eb3-bde18580ca8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:30.518292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5a9830bf-a4c4-4f9b-8eb3-bde18580ca8a None None] MFA Rules not processed for user `9cc891cc8172490394d2bfa3c8d1dcf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:30.519516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5a9830bf-a4c4-4f9b-8eb3-bde18580ca8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:30.531394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:30.575604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1262/2462] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:31.101024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951299be-5480-42b9-ac35-31c856743d33 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:31.175950 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951299be-5480-42b9-ac35-31c856743d33 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:31.181588 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:31.182583 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:31.182944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:31.183331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] PATH_INFO: `/v3/users/dc5a97d65776461f8892b4575754ed12` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:31.184637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5a97d65776461f8892b4575754ed12)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:31.186691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:31.242142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5a97d65776461f8892b4575754ed12'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:31.242685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5a97d65776461f8892b4575754ed12'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:31.243362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5a97d65776461f8892b4575754ed12'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:31.263643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5a97d65776461f8892b4575754ed12'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:31.272851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5a97d65776461f8892b4575754ed12'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:31.288262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1201/2463] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:01:31 2022] DELETE /identity/v3/users/dc5a97d65776461f8892b4575754ed12 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:31.292869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1413800-f109-4fa3-b0cc-6324eff1ece9 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:31.293046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1413800-f109-4fa3-b0cc-6324eff1ece9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:31.293324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1413800-f109-4fa3-b0cc-6324eff1ece9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:31.314357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e1413800-f109-4fa3-b0cc-6324eff1ece9 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:31.315748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e1413800-f109-4fa3-b0cc-6324eff1ece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:31.380410 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1263/2464] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:31.390176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:31.391374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:31.442418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:31.459360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb1493f-925c-48f6-b187-fa561d80f4c4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:31.463652 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918107a-f8d8-49e7-b880-93778e2cac9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:31.464311 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918107a-f8d8-49e7-b880-93778e2cac9f service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:31.464596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918107a-f8d8-49e7-b880-93778e2cac9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:31.464952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918107a-f8d8-49e7-b880-93778e2cac9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:31.476316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918107a-f8d8-49e7-b880-93778e2cac9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:31.477976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918107a-f8d8-49e7-b880-93778e2cac9f service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:31.488469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1202/2465] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:01:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:01:31.765279 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1413800-f109-4fa3-b0cc-6324eff1ece9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:31.768218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e1413800-f109-4fa3-b0cc-6324eff1ece9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:31.781977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1413800-f109-4fa3-b0cc-6324eff1ece9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:31.788066 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d9d755-70d2-4fea-86fe-c6b8eba35842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:31.789051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9d755-70d2-4fea-86fe-c6b8eba35842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:31.789339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9d755-70d2-4fea-86fe-c6b8eba35842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:31.789621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d9d755-70d2-4fea-86fe-c6b8eba35842 admin admin] PATH_INFO: `/v3/projects/7c97d4bfb04f4f79b662ce7e85d48a4b` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:31.790825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d9d755-70d2-4fea-86fe-c6b8eba35842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c97d4bfb04f4f79b662ce7e85d48a4b)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:31.792896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d9d755-70d2-4fea-86fe-c6b8eba35842 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:31.841371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d5d9d755-70d2-4fea-86fe-c6b8eba35842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c97d4bfb04f4f79b662ce7e85d48a4b', 'request_id': 'req-d5d9d755-70d2-4fea-86fe-c6b8eba35842'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:31.849115 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d5d9d755-70d2-4fea-86fe-c6b8eba35842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c97d4bfb04f4f79b662ce7e85d48a4b', 'request_id': 'req-d5d9d755-70d2-4fea-86fe-c6b8eba35842'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:31.854940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d5d9d755-70d2-4fea-86fe-c6b8eba35842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c97d4bfb04f4f79b662ce7e85d48a4b', 'request_id': 'req-d5d9d755-70d2-4fea-86fe-c6b8eba35842'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:31.866268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d5d9d755-70d2-4fea-86fe-c6b8eba35842 admin admin] The token cache is being invalidate because project 7c97d4bfb04f4f79b662ce7e85d48a4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:01:31.866559 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d5d9d755-70d2-4fea-86fe-c6b8eba35842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:31.867242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d5d9d755-70d2-4fea-86fe-c6b8eba35842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:31.868893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1264/2466] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:01:31 2022] DELETE /identity/v3/projects/7c97d4bfb04f4f79b662ce7e85d48a4b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:31.873785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8de09-e43d-453c-859f-8e43c73a55fc None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:31.874069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8de09-e43d-453c-859f-8e43c73a55fc None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:31.874347 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8de09-e43d-453c-859f-8e43c73a55fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:31.891505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b0a8de09-e43d-453c-859f-8e43c73a55fc None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:31.893271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b0a8de09-e43d-453c-859f-8e43c73a55fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:31.951299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1203/2467] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:01:31.959651 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c6dc-9953-473f-b73c-2ee6c7289ae9 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:31.959950 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c6dc-9953-473f-b73c-2ee6c7289ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:31.960240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8c6dc-9953-473f-b73c-2ee6c7289ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:31.981268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-27f8c6dc-9953-473f-b73c-2ee6c7289ae9 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:31.982504 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-27f8c6dc-9953-473f-b73c-2ee6c7289ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:31.990514 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1265/2468] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:31.994705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8de09-e43d-453c-859f-8e43c73a55fc None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:32.004966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8de09-e43d-453c-859f-8e43c73a55fc None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:32.009526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc9dae-2dc4-4a35-bc8b-062c444498de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:32.010172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc9dae-2dc4-4a35-bc8b-062c444498de admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:32.010457 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc9dae-2dc4-4a35-bc8b-062c444498de admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:32.010735 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc9dae-2dc4-4a35-bc8b-062c444498de admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:32.011457 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc9dae-2dc4-4a35-bc8b-062c444498de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:01:32.011767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc9dae-2dc4-4a35-bc8b-062c444498de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:32.013473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc9dae-2dc4-4a35-bc8b-062c444498de admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:32.025637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1204/2469] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:01:32.029864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c6dc-9953-473f-b73c-2ee6c7289ae9 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:32.039162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8c6dc-9953-473f-b73c-2ee6c7289ae9 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:32.044316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ca766-91cc-4462-bbc6-5ed6dd5244a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:32.045079 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ca766-91cc-4462-bbc6-5ed6dd5244a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:32.045369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ca766-91cc-4462-bbc6-5ed6dd5244a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:32.045654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ca766-91cc-4462-bbc6-5ed6dd5244a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:32.046361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ca766-91cc-4462-bbc6-5ed6dd5244a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:32.048262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ca766-91cc-4462-bbc6-5ed6dd5244a0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:32.075835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1266/2470] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:01:32 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:01:32.080770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc9dae-2dc4-4a35-bc8b-062c444498de admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:32.092091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc9dae-2dc4-4a35-bc8b-062c444498de admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:32.158941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a7450-61b3-4f65-8a5e-70b76611948c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:32.159654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a7450-61b3-4f65-8a5e-70b76611948c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:32.159925 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a7450-61b3-4f65-8a5e-70b76611948c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:32.160182 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5a7450-61b3-4f65-8a5e-70b76611948c admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:32.161628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a7450-61b3-4f65-8a5e-70b76611948c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:32.162867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5a7450-61b3-4f65-8a5e-70b76611948c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:32.185636 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1205/2471] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:01:32 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:01:32.190152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ca766-91cc-4462-bbc6-5ed6dd5244a0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:32.203263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ca766-91cc-4462-bbc6-5ed6dd5244a0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:32.209946 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ffe71-2bd3-4cfa-9a0f-c39c390730c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:32.210976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ffe71-2bd3-4cfa-9a0f-c39c390730c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:32.211109 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ffe71-2bd3-4cfa-9a0f-c39c390730c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:32.211593 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ffe71-2bd3-4cfa-9a0f-c39c390730c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:32.213042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ffe71-2bd3-4cfa-9a0f-c39c390730c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:32.214936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ffe71-2bd3-4cfa-9a0f-c39c390730c0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:32.232736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1267/2472] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:01:32.235050 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a7450-61b3-4f65-8a5e-70b76611948c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:32.247254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5a7450-61b3-4f65-8a5e-70b76611948c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:32.253150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d06578e-9579-4114-9445-b91c97e5de5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:32.253826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06578e-9579-4114-9445-b91c97e5de5a admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:32.254108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06578e-9579-4114-9445-b91c97e5de5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:32.254386 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d06578e-9579-4114-9445-b91c97e5de5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:32.255120 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06578e-9579-4114-9445-b91c97e5de5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:32.257264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d06578e-9579-4114-9445-b91c97e5de5a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:32.276828 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1206/2473] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:01:32.282792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ffe71-2bd3-4cfa-9a0f-c39c390730c0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:32.302820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ffe71-2bd3-4cfa-9a0f-c39c390730c0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:32.308073 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3c0d3-c929-49be-b92d-fc8803165dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:32.309005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3c0d3-c929-49be-b92d-fc8803165dad admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:32.309292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3c0d3-c929-49be-b92d-fc8803165dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:32.309570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3c0d3-c929-49be-b92d-fc8803165dad admin admin] PATH_INFO: `/v3/projects/a1bb7903665341c4a2db8eb400bf38f3/users/adea4e85f57e4f5ab73284cd5f792f24/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:32.312372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:32.338511 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3c0d3-c929-49be-b92d-fc8803165dad admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1bb7903665341c4a2db8eb400bf38f3, user_id=adea4e85f57e4f5ab73284cd5f792f24, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:32.341460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3c0d3-c929-49be-b92d-fc8803165dad admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:32.356284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1268/2474] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:01:32 2022] PUT /identity/v3/projects/a1bb7903665341c4a2db8eb400bf38f3/users/adea4e85f57e4f5ab73284cd5f792f24/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:32.367305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d336e7b-f05d-46a4-9b66-33f5496c0ede None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:32.367625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d336e7b-f05d-46a4-9b66-33f5496c0ede None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:32.367912 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d336e7b-f05d-46a4-9b66-33f5496c0ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:32.396044 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7d336e7b-f05d-46a4-9b66-33f5496c0ede None None] MFA Rules not processed for user `adea4e85f57e4f5ab73284cd5f792f24`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:32.397307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7d336e7b-f05d-46a4-9b66-33f5496c0ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:32.438501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:32.466883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1207/2475] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:01:32.478639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3c0d3-c929-49be-b92d-fc8803165dad admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:32.480331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a8d3c0d3-c929-49be-b92d-fc8803165dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:32.542068 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3c0d3-c929-49be-b92d-fc8803165dad admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:32.546360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ded47-2054-4b4f-9397-42644fd4ecd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:32.547101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ded47-2054-4b4f-9397-42644fd4ecd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:32.547396 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ded47-2054-4b4f-9397-42644fd4ecd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:32.547820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ded47-2054-4b4f-9397-42644fd4ecd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:32.558003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ded47-2054-4b4f-9397-42644fd4ecd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:32.559460 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ded47-2054-4b4f-9397-42644fd4ecd5 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:32.570550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1269/2476] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:01:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:01:42.463918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=9 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:42.565878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:51.182426 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d336e7b-f05d-46a4-9b66-33f5496c0ede None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:51.246507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d336e7b-f05d-46a4-9b66-33f5496c0ede None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:51.251068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:51.251937 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:51.252225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:51.252504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] PATH_INFO: `/v3/users/adea4e85f57e4f5ab73284cd5f792f24` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:51.253742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=adea4e85f57e4f5ab73284cd5f792f24)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:51.255706 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:51.302742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adea4e85f57e4f5ab73284cd5f792f24'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:51.303406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adea4e85f57e4f5ab73284cd5f792f24'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:51.304024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adea4e85f57e4f5ab73284cd5f792f24'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:51.315811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adea4e85f57e4f5ab73284cd5f792f24'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:51.322383 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adea4e85f57e4f5ab73284cd5f792f24'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:51.334999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1208/2477] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:01:51 2022] DELETE /identity/v3/users/adea4e85f57e4f5ab73284cd5f792f24 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:51.339512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1e40-9312-484c-9d85-8346abee1887 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:51.340031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1e40-9312-484c-9d85-8346abee1887 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:51.340136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea1e40-9312-484c-9d85-8346abee1887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:51.358307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-01ea1e40-9312-484c-9d85-8346abee1887 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:51.359517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-01ea1e40-9312-484c-9d85-8346abee1887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:51.419676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1270/2478] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:51.430274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:51.431700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:51.490310 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179d59f-4723-476d-ac1b-b3cdad8a59fa admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:51.494106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f77f9-b5e5-45cd-9a0d-61a2ab0b6e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:51.494752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f77f9-b5e5-45cd-9a0d-61a2ab0b6e13 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:51.495042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f77f9-b5e5-45cd-9a0d-61a2ab0b6e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:51.495321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f77f9-b5e5-45cd-9a0d-61a2ab0b6e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:51.505515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f77f9-b5e5-45cd-9a0d-61a2ab0b6e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:51.507250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f77f9-b5e5-45cd-9a0d-61a2ab0b6e13 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:51.518011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1209/2479] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:01:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:01:51.846144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea1e40-9312-484c-9d85-8346abee1887 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:51.847763 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-01ea1e40-9312-484c-9d85-8346abee1887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:51.860795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea1e40-9312-484c-9d85-8346abee1887 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:51.865547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004cb395-8c77-4b4f-b24a-1453cccdaa55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:51.866412 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cb395-8c77-4b4f-b24a-1453cccdaa55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:51.866728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cb395-8c77-4b4f-b24a-1453cccdaa55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:51.867015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004cb395-8c77-4b4f-b24a-1453cccdaa55 admin admin] PATH_INFO: `/v3/projects/a1bb7903665341c4a2db8eb400bf38f3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:51.868263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cb395-8c77-4b4f-b24a-1453cccdaa55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1bb7903665341c4a2db8eb400bf38f3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:51.870212 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004cb395-8c77-4b4f-b24a-1453cccdaa55 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:51.916010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-004cb395-8c77-4b4f-b24a-1453cccdaa55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1bb7903665341c4a2db8eb400bf38f3', 'request_id': 'req-004cb395-8c77-4b4f-b24a-1453cccdaa55'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:51.924613 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-004cb395-8c77-4b4f-b24a-1453cccdaa55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1bb7903665341c4a2db8eb400bf38f3', 'request_id': 'req-004cb395-8c77-4b4f-b24a-1453cccdaa55'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:51.930467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-004cb395-8c77-4b4f-b24a-1453cccdaa55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1bb7903665341c4a2db8eb400bf38f3', 'request_id': 'req-004cb395-8c77-4b4f-b24a-1453cccdaa55'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:51.941390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-004cb395-8c77-4b4f-b24a-1453cccdaa55 admin admin] The token cache is being invalidate because project a1bb7903665341c4a2db8eb400bf38f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:01:51.942250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-004cb395-8c77-4b4f-b24a-1453cccdaa55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:51.943065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-004cb395-8c77-4b4f-b24a-1453cccdaa55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:51.944635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1271/2480] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:01:51 2022] DELETE /identity/v3/projects/a1bb7903665341c4a2db8eb400bf38f3 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:51.949694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cabe7f-92b0-4afa-b8ab-e9627ee5fa6e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:51.949984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cabe7f-92b0-4afa-b8ab-e9627ee5fa6e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:51.950261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cabe7f-92b0-4afa-b8ab-e9627ee5fa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:51.967972 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-97cabe7f-92b0-4afa-b8ab-e9627ee5fa6e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:51.969298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-97cabe7f-92b0-4afa-b8ab-e9627ee5fa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:51.987596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:52.032930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1210/2481] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:52.041856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f437918-f918-45f3-804c-22e7226b8bcc None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.042156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f437918-f918-45f3-804c-22e7226b8bcc None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.042452 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f437918-f918-45f3-804c-22e7226b8bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.060647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1f437918-f918-45f3-804c-22e7226b8bcc None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:52.061901 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1f437918-f918-45f3-804c-22e7226b8bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:52.070369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1272/2482] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:52.074679 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cabe7f-92b0-4afa-b8ab-e9627ee5fa6e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:52.085162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cabe7f-92b0-4afa-b8ab-e9627ee5fa6e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:52.089775 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e623eac-e6dc-46e0-bd40-02b842cd690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:52.090425 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e623eac-e6dc-46e0-bd40-02b842cd690a admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.090713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e623eac-e6dc-46e0-bd40-02b842cd690a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.090994 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e623eac-e6dc-46e0-bd40-02b842cd690a admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.091763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e623eac-e6dc-46e0-bd40-02b842cd690a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:01:52.092075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e623eac-e6dc-46e0-bd40-02b842cd690a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:52.093706 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e623eac-e6dc-46e0-bd40-02b842cd690a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:52.109245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1211/2483] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:01:52.113602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f437918-f918-45f3-804c-22e7226b8bcc None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:52.123086 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f437918-f918-45f3-804c-22e7226b8bcc None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:52.127779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231887c9-6139-4006-a3d8-8b4efc44bc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:52.128476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231887c9-6139-4006-a3d8-8b4efc44bc74 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.128800 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231887c9-6139-4006-a3d8-8b4efc44bc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.129086 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231887c9-6139-4006-a3d8-8b4efc44bc74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.130292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231887c9-6139-4006-a3d8-8b4efc44bc74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:52.132233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231887c9-6139-4006-a3d8-8b4efc44bc74 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:52.158241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1273/2484] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:01:52 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:01:52.162795 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e623eac-e6dc-46e0-bd40-02b842cd690a admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:52.172817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e623eac-e6dc-46e0-bd40-02b842cd690a admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:52.228513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e61a81-febd-4a20-baa8-77734d00145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:52.229214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e61a81-febd-4a20-baa8-77734d00145c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.229478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e61a81-febd-4a20-baa8-77734d00145c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.229756 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e61a81-febd-4a20-baa8-77734d00145c admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.230452 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e61a81-febd-4a20-baa8-77734d00145c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:52.232375 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e61a81-febd-4a20-baa8-77734d00145c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:52.253491 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1212/2485] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:01:52 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:01:52.258364 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231887c9-6139-4006-a3d8-8b4efc44bc74 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:52.269088 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231887c9-6139-4006-a3d8-8b4efc44bc74 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:52.273877 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f1e04-b409-44eb-b9c4-99ec5c6ed395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:52.274988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f1e04-b409-44eb-b9c4-99ec5c6ed395 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.274988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f1e04-b409-44eb-b9c4-99ec5c6ed395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.275220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f1e04-b409-44eb-b9c4-99ec5c6ed395 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.275918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f1e04-b409-44eb-b9c4-99ec5c6ed395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:52.277487 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f1e04-b409-44eb-b9c4-99ec5c6ed395 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:52.289300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1274/2486] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:01:52.293347 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e61a81-febd-4a20-baa8-77734d00145c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:52.304122 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e61a81-febd-4a20-baa8-77734d00145c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:52.308776 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d52e7-75e8-4621-9a15-bb9add2b6eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:52.309431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d52e7-75e8-4621-9a15-bb9add2b6eae admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.309720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d52e7-75e8-4621-9a15-bb9add2b6eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.310001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d52e7-75e8-4621-9a15-bb9add2b6eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.310728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d52e7-75e8-4621-9a15-bb9add2b6eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:52.312332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d52e7-75e8-4621-9a15-bb9add2b6eae admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:52.322930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1213/2487] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:01:52.327032 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f1e04-b409-44eb-b9c4-99ec5c6ed395 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:52.337082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f1e04-b409-44eb-b9c4-99ec5c6ed395 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:52.341695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca0c43-85a4-4354-9f19-340027f645e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:52.342574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca0c43-85a4-4354-9f19-340027f645e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.342863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca0c43-85a4-4354-9f19-340027f645e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.343146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca0c43-85a4-4354-9f19-340027f645e8 admin admin] PATH_INFO: `/v3/projects/0f5f132ce6794f2dade7a635bdcb21b0/users/6fef4ebe0037426fb0dccd1c413a85ba/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.357971 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca0c43-85a4-4354-9f19-340027f645e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f5f132ce6794f2dade7a635bdcb21b0, user_id=6fef4ebe0037426fb0dccd1c413a85ba, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:52.360600 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca0c43-85a4-4354-9f19-340027f645e8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:52.373970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1275/2488] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:01:52 2022] PUT /identity/v3/projects/0f5f132ce6794f2dade7a635bdcb21b0/users/6fef4ebe0037426fb0dccd1c413a85ba/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:52.384841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6518d-bc57-4dc5-a406-adc6190d0913 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.385143 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6518d-bc57-4dc5-a406-adc6190d0913 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.385429 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab6518d-bc57-4dc5-a406-adc6190d0913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.411694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-6ab6518d-bc57-4dc5-a406-adc6190d0913 None None] MFA Rules not processed for user `6fef4ebe0037426fb0dccd1c413a85ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:52.413408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6ab6518d-bc57-4dc5-a406-adc6190d0913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:52.477834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1214/2489] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:52.487695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca0c43-85a4-4354-9f19-340027f645e8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:52.509797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:52.551969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca0c43-85a4-4354-9f19-340027f645e8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:52.557252 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126acdd2-10c0-4e8c-8e67-772e72d2428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:52.557909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126acdd2-10c0-4e8c-8e67-772e72d2428e admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.558201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126acdd2-10c0-4e8c-8e67-772e72d2428e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.558485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126acdd2-10c0-4e8c-8e67-772e72d2428e admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.559181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126acdd2-10c0-4e8c-8e67-772e72d2428e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:52.561126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126acdd2-10c0-4e8c-8e67-772e72d2428e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:52.586055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1276/2490] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:01:52 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:01:52.590157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6518d-bc57-4dc5-a406-adc6190d0913 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:52.601472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab6518d-bc57-4dc5-a406-adc6190d0913 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:52.613441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:52.657303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e409f5-28dc-49bc-8bab-7634215903c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:52.657975 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e409f5-28dc-49bc-8bab-7634215903c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.658266 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e409f5-28dc-49bc-8bab-7634215903c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.658550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e409f5-28dc-49bc-8bab-7634215903c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.659254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e409f5-28dc-49bc-8bab-7634215903c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:52.661205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e409f5-28dc-49bc-8bab-7634215903c1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:52.687177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1215/2491] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:01:52 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:01:52.688405 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126acdd2-10c0-4e8c-8e67-772e72d2428e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:52.698984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126acdd2-10c0-4e8c-8e67-772e72d2428e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:52.703560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5dd41-0021-43ad-958f-c16578520c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:52.704241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5dd41-0021-43ad-958f-c16578520c4a admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.704530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5dd41-0021-43ad-958f-c16578520c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.704855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e5dd41-0021-43ad-958f-c16578520c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.705641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5dd41-0021-43ad-958f-c16578520c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:52.707241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e5dd41-0021-43ad-958f-c16578520c4a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:52.718316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1277/2492] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:01:52.722075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e409f5-28dc-49bc-8bab-7634215903c1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:52.733926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e409f5-28dc-49bc-8bab-7634215903c1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:52.738581 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474af8c-e25a-4997-9dcf-dd314496b76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:52.739260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474af8c-e25a-4997-9dcf-dd314496b76b admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.739542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474af8c-e25a-4997-9dcf-dd314496b76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.739820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474af8c-e25a-4997-9dcf-dd314496b76b admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.741223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474af8c-e25a-4997-9dcf-dd314496b76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:52.742819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474af8c-e25a-4997-9dcf-dd314496b76b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:52.753946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1216/2493] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:01:52.759603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5dd41-0021-43ad-958f-c16578520c4a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:52.771565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e5dd41-0021-43ad-958f-c16578520c4a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:52.776770 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eea804-32f3-44cc-ba71-020494278fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:52.777250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eea804-32f3-44cc-ba71-020494278fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.777535 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eea804-32f3-44cc-ba71-020494278fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.777859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eea804-32f3-44cc-ba71-020494278fd4 admin admin] PATH_INFO: `/v3/projects/b86d3760bb094a2eb6ec42a9689bbeca/users/5f5ee78551cb4c05a715642551b7465d/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.795328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eea804-32f3-44cc-ba71-020494278fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b86d3760bb094a2eb6ec42a9689bbeca, user_id=5f5ee78551cb4c05a715642551b7465d, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:52.798076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eea804-32f3-44cc-ba71-020494278fd4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:52.813695 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1278/2494] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:01:52 2022] PUT /identity/v3/projects/b86d3760bb094a2eb6ec42a9689bbeca/users/5f5ee78551cb4c05a715642551b7465d/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:52.825245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee6ff2-f070-4bb3-a4c1-6a9f5fc123ff None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.825541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee6ff2-f070-4bb3-a4c1-6a9f5fc123ff None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.825849 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee6ff2-f070-4bb3-a4c1-6a9f5fc123ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.853683 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b2ee6ff2-f070-4bb3-a4c1-6a9f5fc123ff None None] MFA Rules not processed for user `5f5ee78551cb4c05a715642551b7465d`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:52.854907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b2ee6ff2-f070-4bb3-a4c1-6a9f5fc123ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:52.920020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1217/2495] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:52.924809 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eea804-32f3-44cc-ba71-020494278fd4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:52.986439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eea804-32f3-44cc-ba71-020494278fd4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:52.991008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e810aa-75a5-40a4-b36c-41833aecf9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:52.991664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e810aa-75a5-40a4-b36c-41833aecf9fc admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:52.991954 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e810aa-75a5-40a4-b36c-41833aecf9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:52.992236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e810aa-75a5-40a4-b36c-41833aecf9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:52.992969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e810aa-75a5-40a4-b36c-41833aecf9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:52.994867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e810aa-75a5-40a4-b36c-41833aecf9fc admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:53.019590 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1279/2496] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:01:52 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:01:53.024417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee6ff2-f070-4bb3-a4c1-6a9f5fc123ff None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:53.035241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee6ff2-f070-4bb3-a4c1-6a9f5fc123ff None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:53.091640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5da45-ba62-49a1-9def-2fc644f24bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:53.092316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5da45-ba62-49a1-9def-2fc644f24bfb admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:53.092607 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5da45-ba62-49a1-9def-2fc644f24bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:53.092924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5da45-ba62-49a1-9def-2fc644f24bfb admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:53.093625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5da45-ba62-49a1-9def-2fc644f24bfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:53.095550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5da45-ba62-49a1-9def-2fc644f24bfb admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:53.115230 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1218/2497] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:01:53 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:01:53.119567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e810aa-75a5-40a4-b36c-41833aecf9fc admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:53.129842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e810aa-75a5-40a4-b36c-41833aecf9fc admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:53.134662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37e6a2-814c-4a98-97f3-694e984bebd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:53.135319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37e6a2-814c-4a98-97f3-694e984bebd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:53.135609 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37e6a2-814c-4a98-97f3-694e984bebd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:53.135892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37e6a2-814c-4a98-97f3-694e984bebd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:53.136926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37e6a2-814c-4a98-97f3-694e984bebd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:53.139491 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d37e6a2-814c-4a98-97f3-694e984bebd5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:53.151003 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1280/2498] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:01:53.155301 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5da45-ba62-49a1-9def-2fc644f24bfb admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:53.166026 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5da45-ba62-49a1-9def-2fc644f24bfb admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:53.170669 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74b107-ea9a-4fb1-883f-03acf85dc0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:53.171597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74b107-ea9a-4fb1-883f-03acf85dc0e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:53.171873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74b107-ea9a-4fb1-883f-03acf85dc0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:53.172173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74b107-ea9a-4fb1-883f-03acf85dc0e1 admin admin] PATH_INFO: `/v3/projects/2a6be87d00e74a2b8e63c5236ca136d0/users/c86037dc16b544fdaf148760d72c6c96/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:53.184445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:53.187807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74b107-ea9a-4fb1-883f-03acf85dc0e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6be87d00e74a2b8e63c5236ca136d0, user_id=c86037dc16b544fdaf148760d72c6c96, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:53.190520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee74b107-ea9a-4fb1-883f-03acf85dc0e1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:53.204087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1219/2499] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:01:53 2022] PUT /identity/v3/projects/2a6be87d00e74a2b8e63c5236ca136d0/users/c86037dc16b544fdaf148760d72c6c96/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:53.208228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37e6a2-814c-4a98-97f3-694e984bebd5 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:53.218842 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d37e6a2-814c-4a98-97f3-694e984bebd5 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:53.263599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:53.281301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfe0e4-8b02-46a9-9f02-6e0acc527f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:53.281968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfe0e4-8b02-46a9-9f02-6e0acc527f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:53.282259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfe0e4-8b02-46a9-9f02-6e0acc527f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:53.282538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfe0e4-8b02-46a9-9f02-6e0acc527f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:53.283257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfe0e4-8b02-46a9-9f02-6e0acc527f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:53.284884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcfe0e4-8b02-46a9-9f02-6e0acc527f73 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:53.295055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1281/2500] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:01:53.299665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74b107-ea9a-4fb1-883f-03acf85dc0e1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:53.309961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74b107-ea9a-4fb1-883f-03acf85dc0e1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:53.314581 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5843ba-21ba-4842-bb7b-b7b62c9b8238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:53.315451 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5843ba-21ba-4842-bb7b-b7b62c9b8238 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:53.315738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5843ba-21ba-4842-bb7b-b7b62c9b8238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:53.316016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5843ba-21ba-4842-bb7b-b7b62c9b8238 admin admin] PATH_INFO: `/v3/projects/2a6be87d00e74a2b8e63c5236ca136d0/users/c86037dc16b544fdaf148760d72c6c96/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:53.318028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5843ba-21ba-4842-bb7b-b7b62c9b8238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a6be87d00e74a2b8e63c5236ca136d0, user_id=c86037dc16b544fdaf148760d72c6c96, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:53.321213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5843ba-21ba-4842-bb7b-b7b62c9b8238 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:53.331247 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1220/2501] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:01:53 2022] PUT /identity/v3/projects/2a6be87d00e74a2b8e63c5236ca136d0/users/c86037dc16b544fdaf148760d72c6c96/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:53.392827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66345184-7f7b-4025-9240-04caba7007b4 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:53.393361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66345184-7f7b-4025-9240-04caba7007b4 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:53.393484 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66345184-7f7b-4025-9240-04caba7007b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:53.420038 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-66345184-7f7b-4025-9240-04caba7007b4 None None] MFA Rules not processed for user `c86037dc16b544fdaf148760d72c6c96`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:53.421365 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-66345184-7f7b-4025-9240-04caba7007b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:53.489507 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1282/2502] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:53.500772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5843ba-21ba-4842-bb7b-b7b62c9b8238 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:53.502538 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ab5843ba-21ba-4842-bb7b-b7b62c9b8238 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:53.560998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5843ba-21ba-4842-bb7b-b7b62c9b8238 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:53.564591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77754d-e6ab-4e48-bc11-eac51214cc64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:53.565258 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77754d-e6ab-4e48-bc11-eac51214cc64 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:53.565549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77754d-e6ab-4e48-bc11-eac51214cc64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:53.565831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77754d-e6ab-4e48-bc11-eac51214cc64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:53.622621 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77754d-e6ab-4e48-bc11-eac51214cc64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:53.624056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77754d-e6ab-4e48-bc11-eac51214cc64 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:53.633585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1221/2503] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:01:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:01:55.131199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66345184-7f7b-4025-9240-04caba7007b4 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:55.132743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-66345184-7f7b-4025-9240-04caba7007b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:55.145183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66345184-7f7b-4025-9240-04caba7007b4 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:55.149221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298e26e-3f70-4ef1-853c-98beb3a86f29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:55.149860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298e26e-3f70-4ef1-853c-98beb3a86f29 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:55.150142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298e26e-3f70-4ef1-853c-98beb3a86f29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:55.150418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298e26e-3f70-4ef1-853c-98beb3a86f29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:55.207922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298e26e-3f70-4ef1-853c-98beb3a86f29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:55.209953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298e26e-3f70-4ef1-853c-98beb3a86f29 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:55.219153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1283/2504] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:01:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:01:55.742803 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77754d-e6ab-4e48-bc11-eac51214cc64 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:55.751813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77754d-e6ab-4e48-bc11-eac51214cc64 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:55.755531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a47a42-eb90-4774-8e90-1fa57052053b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:55.756699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a47a42-eb90-4774-8e90-1fa57052053b service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:55.757115 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a47a42-eb90-4774-8e90-1fa57052053b service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:55.757335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a47a42-eb90-4774-8e90-1fa57052053b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:55.766490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a47a42-eb90-4774-8e90-1fa57052053b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:55.767913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a47a42-eb90-4774-8e90-1fa57052053b service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:55.777932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1222/2505] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:01:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:01:58.515962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb069f-f19a-4fcb-bddd-be294dedf383 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:58.516523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb069f-f19a-4fcb-bddd-be294dedf383 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:58.516652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb069f-f19a-4fcb-bddd-be294dedf383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:58.539867 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-8deb069f-f19a-4fcb-bddd-be294dedf383 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:58.541133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8deb069f-f19a-4fcb-bddd-be294dedf383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:58.605501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1284/2506] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 12:01:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:01:58.618711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a47a42-eb90-4774-8e90-1fa57052053b service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:58.620889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-96a47a42-eb90-4774-8e90-1fa57052053b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:58.699159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a47a42-eb90-4774-8e90-1fa57052053b service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:58.705013 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5bf2f-fd9f-4870-bbe7-4815332ea44c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:58.705676 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5bf2f-fd9f-4870-bbe7-4815332ea44c service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:58.705966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5bf2f-fd9f-4870-bbe7-4815332ea44c service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:58.706287 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5bf2f-fd9f-4870-bbe7-4815332ea44c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:58.715344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5bf2f-fd9f-4870-bbe7-4815332ea44c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:58.717027 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5bf2f-fd9f-4870-bbe7-4815332ea44c service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:58.726472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1223/2507] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 12:01:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 12:01:59.651532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb069f-f19a-4fcb-bddd-be294dedf383 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:01:59.670004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:01:59.732116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb069f-f19a-4fcb-bddd-be294dedf383 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:01:59.736593 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:01:59.737500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:59.737789 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:59.738066 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] PATH_INFO: `/v3/users/6fef4ebe0037426fb0dccd1c413a85ba` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:59.740703 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fef4ebe0037426fb0dccd1c413a85ba)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:01:59.741926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:01:59.829163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fef4ebe0037426fb0dccd1c413a85ba'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:59.830180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fef4ebe0037426fb0dccd1c413a85ba'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:59.831211 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fef4ebe0037426fb0dccd1c413a85ba'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:59.869019 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fef4ebe0037426fb0dccd1c413a85ba'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:59.887592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fef4ebe0037426fb0dccd1c413a85ba'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:01:59.914234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1285/2508] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:01:59 2022] DELETE /identity/v3/users/6fef4ebe0037426fb0dccd1c413a85ba => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:01:59.916536 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e669-2564-4434-aa5c-b622d9843bf3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:01:59.917103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e669-2564-4434-aa5c-b622d9843bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:01:59.917353 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7e669-2564-4434-aa5c-b622d9843bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:01:59.968422 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-23b7e669-2564-4434-aa5c-b622d9843bf3 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:01:59.969780 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-23b7e669-2564-4434-aa5c-b622d9843bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:01:59.974159 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:00.047912 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1224/2509] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:00.058952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:00.060650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:00.126444 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da17c2-22dd-458f-b80e-e8cff36a6e19 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:00.133037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ada31a-acaf-4b2c-8598-8e2da0d77976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:00.134019 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ada31a-acaf-4b2c-8598-8e2da0d77976 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:00.134344 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ada31a-acaf-4b2c-8598-8e2da0d77976 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:00.134580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ada31a-acaf-4b2c-8598-8e2da0d77976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:00.145672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ada31a-acaf-4b2c-8598-8e2da0d77976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:00.147182 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ada31a-acaf-4b2c-8598-8e2da0d77976 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:00.161475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1286/2510] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:02:00.533856 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7e669-2564-4434-aa5c-b622d9843bf3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:00.535315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-23b7e669-2564-4434-aa5c-b622d9843bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:00.563872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b7e669-2564-4434-aa5c-b622d9843bf3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:00.578922 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:00.579791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:00.580078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:00.580352 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] PATH_INFO: `/v3/projects/0f5f132ce6794f2dade7a635bdcb21b0` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:00.581578 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f5f132ce6794f2dade7a635bdcb21b0)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:00.583491 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:00.643300 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f5f132ce6794f2dade7a635bdcb21b0', 'request_id': 'req-62f50dce-f034-422b-9248-1f166426f779'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:00.654368 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f5f132ce6794f2dade7a635bdcb21b0', 'request_id': 'req-62f50dce-f034-422b-9248-1f166426f779'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:00.661644 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f5f132ce6794f2dade7a635bdcb21b0', 'request_id': 'req-62f50dce-f034-422b-9248-1f166426f779'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:00.674817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] The token cache is being invalidate because project 0f5f132ce6794f2dade7a635bdcb21b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:02:00.675366 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:00.676060 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:00.680800 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1225/2511] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:02:00 2022] DELETE /identity/v3/projects/0f5f132ce6794f2dade7a635bdcb21b0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:02:00.682253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ada31a-acaf-4b2c-8598-8e2da0d77976 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:00.696574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ada31a-acaf-4b2c-8598-8e2da0d77976 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:00.741551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:00.781008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:00.782204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:00.782486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:00.782764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] PATH_INFO: `/v3/users/5f5ee78551cb4c05a715642551b7465d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:00.784002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f5ee78551cb4c05a715642551b7465d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:00.785974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:00.851189 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f5ee78551cb4c05a715642551b7465d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:00.852269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f5ee78551cb4c05a715642551b7465d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:00.852936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f5ee78551cb4c05a715642551b7465d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:00.868062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f5ee78551cb4c05a715642551b7465d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:00.879294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f5ee78551cb4c05a715642551b7465d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:00.901134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1287/2512] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:02:00 2022] DELETE /identity/v3/users/5f5ee78551cb4c05a715642551b7465d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:02:01.291074 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:01.293696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:01.310312 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f50dce-f034-422b-9248-1f166426f779 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:01.378754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:01.379648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:01.379927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:01.380200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] PATH_INFO: `/v3/projects/b86d3760bb094a2eb6ec42a9689bbeca` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:01.381717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b86d3760bb094a2eb6ec42a9689bbeca)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:01.383723 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:01.426973 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:01.440069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b86d3760bb094a2eb6ec42a9689bbeca', 'request_id': 'req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:01.452602 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b86d3760bb094a2eb6ec42a9689bbeca', 'request_id': 'req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:01.463978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b86d3760bb094a2eb6ec42a9689bbeca', 'request_id': 'req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:01.478551 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] The token cache is being invalidate because project b86d3760bb094a2eb6ec42a9689bbeca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:02:01.478852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:01.479563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:01.481252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1226/2513] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:02:01 2022] DELETE /identity/v3/projects/b86d3760bb094a2eb6ec42a9689bbeca => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:02:01.488861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:01.490418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:01.504382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1183d-cfc5-4ee9-8d19-c81261f58b63 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:01.582779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4787ab-e0bd-4f05-8a27-73965a453efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:01.583689 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4787ab-e0bd-4f05-8a27-73965a453efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:01.584559 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4787ab-e0bd-4f05-8a27-73965a453efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:01.584873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4787ab-e0bd-4f05-8a27-73965a453efb admin admin] PATH_INFO: `/v3/users/c86037dc16b544fdaf148760d72c6c96` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:01.586128 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4787ab-e0bd-4f05-8a27-73965a453efb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c86037dc16b544fdaf148760d72c6c96)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:01.588111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4787ab-e0bd-4f05-8a27-73965a453efb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:01.643483 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7b4787ab-e0bd-4f05-8a27-73965a453efb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c86037dc16b544fdaf148760d72c6c96'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:01.644134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7b4787ab-e0bd-4f05-8a27-73965a453efb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c86037dc16b544fdaf148760d72c6c96'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:01.644856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7b4787ab-e0bd-4f05-8a27-73965a453efb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c86037dc16b544fdaf148760d72c6c96'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:01.654122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:01.679376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7b4787ab-e0bd-4f05-8a27-73965a453efb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c86037dc16b544fdaf148760d72c6c96'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:01.690374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-7b4787ab-e0bd-4f05-8a27-73965a453efb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c86037dc16b544fdaf148760d72c6c96'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:01.706663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1288/2514] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:02:01 2022] DELETE /identity/v3/users/c86037dc16b544fdaf148760d72c6c96 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:02:02.073139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:02.075990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:02.103738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0c574-4ff7-4d28-a06d-fee36ecd2e80 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:02.179553 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:02.180457 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:02.180778 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:02.181112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae admin admin] PATH_INFO: `/v3/projects/2a6be87d00e74a2b8e63c5236ca136d0` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:02.182413 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a6be87d00e74a2b8e63c5236ca136d0)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:02.184600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:02.277720 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a6be87d00e74a2b8e63c5236ca136d0', 'request_id': 'req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:02.297334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a6be87d00e74a2b8e63c5236ca136d0', 'request_id': 'req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:02.309370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a6be87d00e74a2b8e63c5236ca136d0', 'request_id': 'req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:02.329466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae admin admin] The token cache is being invalidate because project 2a6be87d00e74a2b8e63c5236ca136d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:02:02.330006 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:02.330884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f7fe1cc0-f42a-4c55-9296-3448a79e3eae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:02.337900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1227/2515] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:02:02 2022] DELETE /identity/v3/projects/2a6be87d00e74a2b8e63c5236ca136d0 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:02:02.338190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311ba43-d790-4562-bea1-a0a654875a0c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:02.339078 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311ba43-d790-4562-bea1-a0a654875a0c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:02.339652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311ba43-d790-4562-bea1-a0a654875a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:02.367536 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f311ba43-d790-4562-bea1-a0a654875a0c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:02.369150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f311ba43-d790-4562-bea1-a0a654875a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:02.454221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1289/2516] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:02.466119 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cacb63-22e9-4081-abd3-76beed8b25f5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:02.466409 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cacb63-22e9-4081-abd3-76beed8b25f5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:02.466689 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cacb63-22e9-4081-abd3-76beed8b25f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:02.491038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-08cacb63-22e9-4081-abd3-76beed8b25f5 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:02.492478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-08cacb63-22e9-4081-abd3-76beed8b25f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:02.501882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1228/2517] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:02.511002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311ba43-d790-4562-bea1-a0a654875a0c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:02.527960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311ba43-d790-4562-bea1-a0a654875a0c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:02.533577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182957c-e9ec-42af-865d-5c35ab210b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:02.534445 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182957c-e9ec-42af-865d-5c35ab210b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:02.534744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182957c-e9ec-42af-865d-5c35ab210b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:02.535029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182957c-e9ec-42af-865d-5c35ab210b37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:02.537117 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182957c-e9ec-42af-865d-5c35ab210b37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:02:02.537442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182957c-e9ec-42af-865d-5c35ab210b37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:02.539108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182957c-e9ec-42af-865d-5c35ab210b37 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:02.552720 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1290/2518] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 12:02:02.556876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cacb63-22e9-4081-abd3-76beed8b25f5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:02.567240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cacb63-22e9-4081-abd3-76beed8b25f5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:02.574039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fe92c-2dff-4469-b69f-d8b393bb14f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:02.574834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fe92c-2dff-4469-b69f-d8b393bb14f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:02.575193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fe92c-2dff-4469-b69f-d8b393bb14f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:02.575545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fe92c-2dff-4469-b69f-d8b393bb14f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:02.576328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fe92c-2dff-4469-b69f-d8b393bb14f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:02.578525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fe92c-2dff-4469-b69f-d8b393bb14f9 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:02.616759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1229/2519] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:02:02 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:02:02.617102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182957c-e9ec-42af-865d-5c35ab210b37 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:02.628253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182957c-e9ec-42af-865d-5c35ab210b37 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:02.697831 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20985c46-7db3-4f4f-a549-a88cd856cf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:02.698558 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20985c46-7db3-4f4f-a549-a88cd856cf02 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:02.698800 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20985c46-7db3-4f4f-a549-a88cd856cf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:02.699087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20985c46-7db3-4f4f-a549-a88cd856cf02 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:02.699820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20985c46-7db3-4f4f-a549-a88cd856cf02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:02.701812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20985c46-7db3-4f4f-a549-a88cd856cf02 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:02.728543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1291/2520] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:02:02 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:02:02.732693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fe92c-2dff-4469-b69f-d8b393bb14f9 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:02.746852 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fe92c-2dff-4469-b69f-d8b393bb14f9 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:02.751686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641307a-1745-4c97-bfd2-dbf1cc770078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:02.752351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641307a-1745-4c97-bfd2-dbf1cc770078 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:02.752643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641307a-1745-4c97-bfd2-dbf1cc770078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:02.752960 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641307a-1745-4c97-bfd2-dbf1cc770078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:02.753697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3641307a-1745-4c97-bfd2-dbf1cc770078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:02.755402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3641307a-1745-4c97-bfd2-dbf1cc770078 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:02.767549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1230/2521] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:02:02.772429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20985c46-7db3-4f4f-a549-a88cd856cf02 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:02.782953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20985c46-7db3-4f4f-a549-a88cd856cf02 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:02.787952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31554c98-c9f8-4065-a1eb-96d47c9de5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:02.788623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31554c98-c9f8-4065-a1eb-96d47c9de5eb admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:02.790260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31554c98-c9f8-4065-a1eb-96d47c9de5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:02.790548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31554c98-c9f8-4065-a1eb-96d47c9de5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:02.791299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31554c98-c9f8-4065-a1eb-96d47c9de5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:02.792991 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31554c98-c9f8-4065-a1eb-96d47c9de5eb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:02.806122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1292/2522] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:02:02.810036 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641307a-1745-4c97-bfd2-dbf1cc770078 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:02.826404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641307a-1745-4c97-bfd2-dbf1cc770078 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:02.831381 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3659d7-11a1-4cea-9e0a-a5db332f1be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:02.832274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3659d7-11a1-4cea-9e0a-a5db332f1be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:02.832558 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3659d7-11a1-4cea-9e0a-a5db332f1be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:02.832901 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3659d7-11a1-4cea-9e0a-a5db332f1be3 admin admin] PATH_INFO: `/v3/projects/a01b251cf9984df393557d138bfbd921/users/cadd12c34c844bf5b33539375fc17350/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:02.847360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:02.852208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3659d7-11a1-4cea-9e0a-a5db332f1be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a01b251cf9984df393557d138bfbd921, user_id=cadd12c34c844bf5b33539375fc17350, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:02.856224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3659d7-11a1-4cea-9e0a-a5db332f1be3 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:02.874637 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1231/2523] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:02:02 2022] PUT /identity/v3/projects/a01b251cf9984df393557d138bfbd921/users/cadd12c34c844bf5b33539375fc17350/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:02:02.886892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9ff06-26e6-45b1-8b59-67d64db4b058 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:02.887191 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9ff06-26e6-45b1-8b59-67d64db4b058 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:02.887503 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f9ff06-26e6-45b1-8b59-67d64db4b058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:02.921348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-e8f9ff06-26e6-45b1-8b59-67d64db4b058 None None] MFA Rules not processed for user `cadd12c34c844bf5b33539375fc17350`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:02.922601 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e8f9ff06-26e6-45b1-8b59-67d64db4b058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:02.957564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:03.010015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1293/2524] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:02:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:03.021265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3659d7-11a1-4cea-9e0a-a5db332f1be3 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:03.023333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5d3659d7-11a1-4cea-9e0a-a5db332f1be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:03.108045 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3659d7-11a1-4cea-9e0a-a5db332f1be3 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:03.112910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc94e45-5e4c-471a-995a-169d15dc47a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74fdc12d728d45018ea668a4073da3b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:03.113714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc94e45-5e4c-471a-995a-169d15dc47a2 service glance] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:03.114071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc94e45-5e4c-471a-995a-169d15dc47a2 service glance] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:03.114419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc94e45-5e4c-471a-995a-169d15dc47a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:03.126906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc94e45-5e4c-471a-995a-169d15dc47a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:03.128464 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc94e45-5e4c-471a-995a-169d15dc47a2 service glance] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:03.138940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1232/2525] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:02:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:02:13.003735 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=16 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:13.134028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:14.810527 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e149b47-a894-463f-ac91-d8110a49dd3c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:14.811053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e149b47-a894-463f-ac91-d8110a49dd3c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:14.811053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e149b47-a894-463f-ac91-d8110a49dd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:14.828762 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-2e149b47-a894-463f-ac91-d8110a49dd3c None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:14.829977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2e149b47-a894-463f-ac91-d8110a49dd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:14.892213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1294/2526] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 23 12:02:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:14.906303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc94e45-5e4c-471a-995a-169d15dc47a2 service glance] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:14.908262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-dbc94e45-5e4c-471a-995a-169d15dc47a2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:14.994336 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc94e45-5e4c-471a-995a-169d15dc47a2 service glance] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:14.999387 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50889680-5c63-4a2b-9953-3a95c2d72b62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:15.000030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50889680-5c63-4a2b-9953-3a95c2d72b62 service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:15.000320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50889680-5c63-4a2b-9953-3a95c2d72b62 service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:15.000603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50889680-5c63-4a2b-9953-3a95c2d72b62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:15.009777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50889680-5c63-4a2b-9953-3a95c2d72b62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:15.011367 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50889680-5c63-4a2b-9953-3a95c2d72b62 service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:15.019785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1233/2527] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 12:02:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 12:02:24.884560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=25 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:25.016513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=32 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:34.280522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024ed76-c9fb-4479-a030-4fcb43043418 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:34.281011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024ed76-c9fb-4479-a030-4fcb43043418 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:34.281207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c024ed76-c9fb-4479-a030-4fcb43043418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:34.299472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c024ed76-c9fb-4479-a030-4fcb43043418 None None] MFA Rules not processed for user `646af794697941d6a46d76086e7e8ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:34.300990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c024ed76-c9fb-4479-a030-4fcb43043418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:34.308471 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1295/2528] 10.222.0.67 () {58 vars in 967 bytes} [Thu Jun 23 12:02:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:34.322186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50889680-5c63-4a2b-9953-3a95c2d72b62 service swift] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:34.333661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50889680-5c63-4a2b-9953-3a95c2d72b62 service swift] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:34.339883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb80a76-bd39-4fe5-9c4b-1c977b5ae29a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb9b9af5e0b44e34837e1cdb5ff6d33e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:34.340562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb80a76-bd39-4fe5-9c4b-1c977b5ae29a service swift] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:34.340890 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb80a76-bd39-4fe5-9c4b-1c977b5ae29a service swift] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:34.341180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb80a76-bd39-4fe5-9c4b-1c977b5ae29a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:34.351605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb80a76-bd39-4fe5-9c4b-1c977b5ae29a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:34.353871 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb80a76-bd39-4fe5-9c4b-1c977b5ae29a service swift] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:34.373660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1234/2529] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 23 12:02:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 12:02:37.937394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024ed76-c9fb-4479-a030-4fcb43043418 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:38.001628 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c024ed76-c9fb-4479-a030-4fcb43043418 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:38.006353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:38.007237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:38.007815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:38.007815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] PATH_INFO: `/v3/users/cadd12c34c844bf5b33539375fc17350` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:38.009715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cadd12c34c844bf5b33539375fc17350)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:38.011678 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:38.058221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cadd12c34c844bf5b33539375fc17350'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:38.058893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cadd12c34c844bf5b33539375fc17350'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:38.059526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cadd12c34c844bf5b33539375fc17350'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:38.072548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cadd12c34c844bf5b33539375fc17350'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:38.079519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cadd12c34c844bf5b33539375fc17350'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:38.093313 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1296/2530] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:02:37 2022] DELETE /identity/v3/users/cadd12c34c844bf5b33539375fc17350 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:02:38.097970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29093de9-09e8-4923-b995-f2a2d7754cc4 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:38.098669 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29093de9-09e8-4923-b995-f2a2d7754cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:38.098883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29093de9-09e8-4923-b995-f2a2d7754cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:38.117578 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-29093de9-09e8-4923-b995-f2a2d7754cc4 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:38.118825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-29093de9-09e8-4923-b995-f2a2d7754cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:38.187896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1235/2531] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:02:38.198871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:38.201061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:38.267904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9382e211-d98c-4aad-8e7d-49406206aaa6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:38.271554 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f026a32-6b9a-4412-9059-a1bfc6095460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:38.272196 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f026a32-6b9a-4412-9059-a1bfc6095460 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:38.272480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f026a32-6b9a-4412-9059-a1bfc6095460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:38.272786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f026a32-6b9a-4412-9059-a1bfc6095460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:38.284521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f026a32-6b9a-4412-9059-a1bfc6095460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:38.285997 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f026a32-6b9a-4412-9059-a1bfc6095460 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:38.296979 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1297/2532] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:02:38.600435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29093de9-09e8-4923-b995-f2a2d7754cc4 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:38.601976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-29093de9-09e8-4923-b995-f2a2d7754cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:38.623073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29093de9-09e8-4923-b995-f2a2d7754cc4 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:38.631927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:38.633443 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:38.634536 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:38.635063 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] PATH_INFO: `/v3/projects/a01b251cf9984df393557d138bfbd921` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:38.638223 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a01b251cf9984df393557d138bfbd921)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:38.640878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:38.693184 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a01b251cf9984df393557d138bfbd921', 'request_id': 'req-dd599c25-e89c-48f1-9687-c8550781e190'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:38.701357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a01b251cf9984df393557d138bfbd921', 'request_id': 'req-dd599c25-e89c-48f1-9687-c8550781e190'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:38.707911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a01b251cf9984df393557d138bfbd921', 'request_id': 'req-dd599c25-e89c-48f1-9687-c8550781e190'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:38.719928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] The token cache is being invalidate because project a01b251cf9984df393557d138bfbd921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:02:38.720206 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:38.720891 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:38.722471 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1236/2533] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:02:38 2022] DELETE /identity/v3/projects/a01b251cf9984df393557d138bfbd921 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:02:43.218968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f026a32-6b9a-4412-9059-a1bfc6095460 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:43.228219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f026a32-6b9a-4412-9059-a1bfc6095460 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:43.239682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:43.279875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7949c-064d-4af3-866e-f9543384f358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:43.280523 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7949c-064d-4af3-866e-f9543384f358 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:43.280853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7949c-064d-4af3-866e-f9543384f358 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:43.281135 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7949c-064d-4af3-866e-f9543384f358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:43.284049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bee7949c-064d-4af3-866e-f9543384f358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:43.353935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7949c-064d-4af3-866e-f9543384f358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:43.356236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7949c-064d-4af3-866e-f9543384f358 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:43.365960 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1298/2534] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:02:45.444820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:45.447478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:45.530039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd599c25-e89c-48f1-9687-c8550781e190 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:45.535199 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa15dff-703c-431b-a954-944383d73efa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:45.535857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa15dff-703c-431b-a954-944383d73efa service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:45.536149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa15dff-703c-431b-a954-944383d73efa service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:45.536435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa15dff-703c-431b-a954-944383d73efa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:45.538362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ffa15dff-703c-431b-a954-944383d73efa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:45.551354 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa15dff-703c-431b-a954-944383d73efa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:45.553011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa15dff-703c-431b-a954-944383d73efa service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:45.559733 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:45.566290 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1237/2535] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:02:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:02:46.896319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7949c-064d-4af3-866e-f9543384f358 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:46.897816 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bee7949c-064d-4af3-866e-f9543384f358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:46.910649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7949c-064d-4af3-866e-f9543384f358 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:46.916629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73710784-0eaf-40dd-9440-0aa88d44e2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:46.918076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73710784-0eaf-40dd-9440-0aa88d44e2e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:46.918846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73710784-0eaf-40dd-9440-0aa88d44e2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:46.919118 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73710784-0eaf-40dd-9440-0aa88d44e2e8 admin admin] PATH_INFO: `/v3/users/9ab84989dd384d15b316b20cc65e792a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:46.920308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73710784-0eaf-40dd-9440-0aa88d44e2e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab84989dd384d15b316b20cc65e792a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:46.922380 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73710784-0eaf-40dd-9440-0aa88d44e2e8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:46.987199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-73710784-0eaf-40dd-9440-0aa88d44e2e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab84989dd384d15b316b20cc65e792a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:46.987199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-73710784-0eaf-40dd-9440-0aa88d44e2e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab84989dd384d15b316b20cc65e792a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:46.987199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-73710784-0eaf-40dd-9440-0aa88d44e2e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab84989dd384d15b316b20cc65e792a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:46.999955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-73710784-0eaf-40dd-9440-0aa88d44e2e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab84989dd384d15b316b20cc65e792a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:47.007246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-73710784-0eaf-40dd-9440-0aa88d44e2e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab84989dd384d15b316b20cc65e792a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:47.020585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1299/2536] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:02:46 2022] DELETE /identity/v3/users/9ab84989dd384d15b316b20cc65e792a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:02:47.021981 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8e85f-901f-474a-88ae-496e42d53460 None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:47.022441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8e85f-901f-474a-88ae-496e42d53460 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:47.022654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8e85f-901f-474a-88ae-496e42d53460 None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:47.024622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1238/2537] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 12:02:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 12:02:47.037010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca0b80-7e6a-4e46-898e-45d1334505b6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:47.037225 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca0b80-7e6a-4e46-898e-45d1334505b6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:47.037509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca0b80-7e6a-4e46-898e-45d1334505b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:47.059304 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-24ca0b80-7e6a-4e46-898e-45d1334505b6 None None] MFA Rules not processed for user `5f095d3370be457cb6687ac8f8c9f8ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:47.060711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-24ca0b80-7e6a-4e46-898e-45d1334505b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:47.098904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:47.130156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1300/2538] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 12:02:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:47.145977 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8e85f-901f-474a-88ae-496e42d53460 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:47.147864 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6ea8e85f-901f-474a-88ae-496e42d53460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:47.162373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8e85f-901f-474a-88ae-496e42d53460 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:47.176295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c41ca-d09d-4d46-8ccc-ef534b2e91f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:47.176944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c41ca-d09d-4d46-8ccc-ef534b2e91f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:47.177232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c41ca-d09d-4d46-8ccc-ef534b2e91f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:47.177514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c41ca-d09d-4d46-8ccc-ef534b2e91f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:47.185938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c41ca-d09d-4d46-8ccc-ef534b2e91f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:47.187376 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c41ca-d09d-4d46-8ccc-ef534b2e91f2 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:47.196597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1239/2539] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:02:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:02:47.506783 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca0b80-7e6a-4e46-898e-45d1334505b6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:47.508438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-24ca0b80-7e6a-4e46-898e-45d1334505b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:47.590150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c7eb6-7580-4b3e-95b7-741457a30c2c None None] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:47.590445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c7eb6-7580-4b3e-95b7-741457a30c2c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:47.590730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c7eb6-7580-4b3e-95b7-741457a30c2c None None] PATH_INFO: `/` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:47.592110 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1240/2540] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 23 12:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 12:02:47.595565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca0b80-7e6a-4e46-898e-45d1334505b6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:47.600269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e2a65-3c35-4f7a-a188-51ef2d9af7c7 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:47.600635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e2a65-3c35-4f7a-a188-51ef2d9af7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:47.600962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e2a65-3c35-4f7a-a188-51ef2d9af7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:47.601291 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d771b9-0db5-4d24-97bb-176288cb6f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:47.602936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d771b9-0db5-4d24-97bb-176288cb6f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:47.603324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d771b9-0db5-4d24-97bb-176288cb6f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:47.603812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d771b9-0db5-4d24-97bb-176288cb6f4a admin admin] PATH_INFO: `/v3/projects/a1a92a7ffc864256b48a9848b27f9fd2` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:47.605470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d771b9-0db5-4d24-97bb-176288cb6f4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1a92a7ffc864256b48a9848b27f9fd2)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:47.608458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d771b9-0db5-4d24-97bb-176288cb6f4a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:47.622491 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ab4e2a65-3c35-4f7a-a188-51ef2d9af7c7 None None] MFA Rules not processed for user `36c18bbd22f74efbae7aab1cdd3f6403`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:47.624483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ab4e2a65-3c35-4f7a-a188-51ef2d9af7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:47.687450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e4d771b9-0db5-4d24-97bb-176288cb6f4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1a92a7ffc864256b48a9848b27f9fd2', 'request_id': 'req-e4d771b9-0db5-4d24-97bb-176288cb6f4a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:47.698106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e4d771b9-0db5-4d24-97bb-176288cb6f4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1a92a7ffc864256b48a9848b27f9fd2', 'request_id': 'req-e4d771b9-0db5-4d24-97bb-176288cb6f4a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:47.706831 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e4d771b9-0db5-4d24-97bb-176288cb6f4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1a92a7ffc864256b48a9848b27f9fd2', 'request_id': 'req-e4d771b9-0db5-4d24-97bb-176288cb6f4a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:47.718656 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e4d771b9-0db5-4d24-97bb-176288cb6f4a admin admin] The token cache is being invalidate because project a1a92a7ffc864256b48a9848b27f9fd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:02:47.718957 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e4d771b9-0db5-4d24-97bb-176288cb6f4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:47.719602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e4d771b9-0db5-4d24-97bb-176288cb6f4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:47.721275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1301/2541] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:02:47 2022] DELETE /identity/v3/projects/a1a92a7ffc864256b48a9848b27f9fd2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:02:47.725905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b5ed6-4a23-4122-af66-d4c2fde1c607 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:47.726240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b5ed6-4a23-4122-af66-d4c2fde1c607 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:47.726486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b5ed6-4a23-4122-af66-d4c2fde1c607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:47.727003 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1241/2542] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 23 12:02:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:02:47.737782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e2a65-3c35-4f7a-a188-51ef2d9af7c7 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:47.746462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-898b5ed6-4a23-4122-af66-d4c2fde1c607 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:47.747292 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e2a65-3c35-4f7a-a188-51ef2d9af7c7 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:47.747698 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-898b5ed6-4a23-4122-af66-d4c2fde1c607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:47.797843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e06ff6-2562-4a9b-8a06-e5279819abc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:47.798399 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e06ff6-2562-4a9b-8a06-e5279819abc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:47.798732 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e06ff6-2562-4a9b-8a06-e5279819abc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:47.799012 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e06ff6-2562-4a9b-8a06-e5279819abc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:47.801106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-06e06ff6-2562-4a9b-8a06-e5279819abc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:47.808781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1302/2543] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:47.816428 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d2bed-ab65-470d-b06a-e3f4d5d04d61 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:47.816749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d2bed-ab65-470d-b06a-e3f4d5d04d61 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:47.817040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d2bed-ab65-470d-b06a-e3f4d5d04d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:47.835664 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-067d2bed-ab65-470d-b06a-e3f4d5d04d61 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:47.837580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-067d2bed-ab65-470d-b06a-e3f4d5d04d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:47.845788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1303/2544] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:02:47.849907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d2bed-ab65-470d-b06a-e3f4d5d04d61 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:47.859579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d2bed-ab65-470d-b06a-e3f4d5d04d61 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:47.864379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1030c-4d20-4a9b-b1c0-3bc80c9be5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:47.865053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1030c-4d20-4a9b-b1c0-3bc80c9be5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:47.865346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1030c-4d20-4a9b-b1c0-3bc80c9be5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:47.865650 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc1030c-4d20-4a9b-b1c0-3bc80c9be5a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:47.866380 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1030c-4d20-4a9b-b1c0-3bc80c9be5a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:02:47.866684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1030c-4d20-4a9b-b1c0-3bc80c9be5a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:47.868335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc1030c-4d20-4a9b-b1c0-3bc80c9be5a6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:47.875962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:47.877129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e06ff6-2562-4a9b-8a06-e5279819abc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:47.878708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e06ff6-2562-4a9b-8a06-e5279819abc8 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:47.882248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1304/2545] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 12:02:47.886471 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1030c-4d20-4a9b-b1c0-3bc80c9be5a6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:47.890050 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1242/2546] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:02:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:02:47.896527 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc1030c-4d20-4a9b-b1c0-3bc80c9be5a6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:47.901862 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944597da-c276-459f-b177-33d6a89697aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:47.902511 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944597da-c276-459f-b177-33d6a89697aa admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:47.902890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944597da-c276-459f-b177-33d6a89697aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:47.903075 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944597da-c276-459f-b177-33d6a89697aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:47.903766 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944597da-c276-459f-b177-33d6a89697aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:47.905709 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944597da-c276-459f-b177-33d6a89697aa admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:47.939915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1305/2547] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:02:47 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:02:47.944920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e06ff6-2562-4a9b-8a06-e5279819abc8 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:47.965933 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e06ff6-2562-4a9b-8a06-e5279819abc8 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:47.987307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:48.038990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce027-939e-4965-89d4-982df81a9da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:48.040284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce027-939e-4965-89d4-982df81a9da5 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:48.041092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce027-939e-4965-89d4-982df81a9da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:48.041445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce027-939e-4965-89d4-982df81a9da5 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:48.042350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ce027-939e-4965-89d4-982df81a9da5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:48.045571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ce027-939e-4965-89d4-982df81a9da5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:48.077485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1243/2548] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:02:47 2022] POST /identity/v3/users => generated 284 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:02:48.079351 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944597da-c276-459f-b177-33d6a89697aa admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:48.091551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944597da-c276-459f-b177-33d6a89697aa admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:48.097999 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6079b-046f-49f7-ac1c-66a47c17d5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:48.098806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6079b-046f-49f7-ac1c-66a47c17d5bb admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:48.098959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6079b-046f-49f7-ac1c-66a47c17d5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:48.099252 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6079b-046f-49f7-ac1c-66a47c17d5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:48.099989 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6079b-046f-49f7-ac1c-66a47c17d5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:48.101794 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6079b-046f-49f7-ac1c-66a47c17d5bb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:48.116502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1306/2549] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:02:48.124931 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce027-939e-4965-89d4-982df81a9da5 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:48.135997 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce027-939e-4965-89d4-982df81a9da5 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:48.141751 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7872f2-1f7a-435b-8aed-4cecc19c7cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:48.142415 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7872f2-1f7a-435b-8aed-4cecc19c7cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:48.142690 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7872f2-1f7a-435b-8aed-4cecc19c7cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:48.142975 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7872f2-1f7a-435b-8aed-4cecc19c7cb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:48.143716 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7872f2-1f7a-435b-8aed-4cecc19c7cb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:48.145410 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7872f2-1f7a-435b-8aed-4cecc19c7cb4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:48.156963 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1244/2550] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:02:48.161058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6079b-046f-49f7-ac1c-66a47c17d5bb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:48.172257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6079b-046f-49f7-ac1c-66a47c17d5bb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:48.179029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6ece3-c6b1-4933-85c7-abf1035dedcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:48.180077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6ece3-c6b1-4933-85c7-abf1035dedcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:48.181113 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6ece3-c6b1-4933-85c7-abf1035dedcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:48.181492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6ece3-c6b1-4933-85c7-abf1035dedcd admin admin] PATH_INFO: `/v3/projects/633f5ccb45f64ba9a716f1a572198866/users/77a5642fab864efabb8324a3dd693db4/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:48.201346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6ece3-c6b1-4933-85c7-abf1035dedcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=633f5ccb45f64ba9a716f1a572198866, user_id=77a5642fab864efabb8324a3dd693db4, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:48.203201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6ece3-c6b1-4933-85c7-abf1035dedcd admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:48.218755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1307/2551] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:02:48 2022] PUT /identity/v3/projects/633f5ccb45f64ba9a716f1a572198866/users/77a5642fab864efabb8324a3dd693db4/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:02:48.229695 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292d2c9-a77e-44e0-84d5-d29fa4843548 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:48.230221 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292d2c9-a77e-44e0-84d5-d29fa4843548 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:48.230664 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292d2c9-a77e-44e0-84d5-d29fa4843548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:48.281561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7292d2c9-a77e-44e0-84d5-d29fa4843548 None None] MFA Rules not processed for user `77a5642fab864efabb8324a3dd693db4`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:48.282894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7292d2c9-a77e-44e0-84d5-d29fa4843548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:48.362479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1245/2552] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:02:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:48.374912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6ece3-c6b1-4933-85c7-abf1035dedcd admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:48.376588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3df6ece3-c6b1-4933-85c7-abf1035dedcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:48.445982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6ece3-c6b1-4933-85c7-abf1035dedcd admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:48.452190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c60f1c-df40-4ed2-ae3f-ede1df6a2344 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:48.453009 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c60f1c-df40-4ed2-ae3f-ede1df6a2344 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:48.453128 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c60f1c-df40-4ed2-ae3f-ede1df6a2344 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:48.453416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c60f1c-df40-4ed2-ae3f-ede1df6a2344 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:48.463654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c60f1c-df40-4ed2-ae3f-ede1df6a2344 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:48.465156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c60f1c-df40-4ed2-ae3f-ede1df6a2344 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:48.474706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1308/2553] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:02:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:02:55.867592 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292d2c9-a77e-44e0-84d5-d29fa4843548 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:55.869064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7292d2c9-a77e-44e0-84d5-d29fa4843548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:55.928188 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292d2c9-a77e-44e0-84d5-d29fa4843548 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:55.932458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d64ef-699f-4601-b4da-6e5ceebdecdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:55.933270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d64ef-699f-4601-b4da-6e5ceebdecdf service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:55.933502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d64ef-699f-4601-b4da-6e5ceebdecdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:55.933781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d64ef-699f-4601-b4da-6e5ceebdecdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:55.935914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cf7d64ef-699f-4601-b4da-6e5ceebdecdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:55.982928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:56.006424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d64ef-699f-4601-b4da-6e5ceebdecdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:56.007861 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d64ef-699f-4601-b4da-6e5ceebdecdf service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:56.018313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1246/2554] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:02:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:02:56.847777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c60f1c-df40-4ed2-ae3f-ede1df6a2344 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:56.859728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c60f1c-df40-4ed2-ae3f-ede1df6a2344 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:56.864531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:56.865577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:56.865965 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:56.866339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] PATH_INFO: `/v3/users/77a5642fab864efabb8324a3dd693db4` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:56.869775 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a5642fab864efabb8324a3dd693db4)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:56.871720 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:56.937121 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a5642fab864efabb8324a3dd693db4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:56.937790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a5642fab864efabb8324a3dd693db4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:56.938403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a5642fab864efabb8324a3dd693db4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:56.954287 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a5642fab864efabb8324a3dd693db4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:56.961850 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a5642fab864efabb8324a3dd693db4'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:56.979340 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1309/2555] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:02:56 2022] DELETE /identity/v3/users/77a5642fab864efabb8324a3dd693db4 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:02:56.983830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54bd7e-5a1b-4eb5-a634-230659c2dc15 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:56.984341 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54bd7e-5a1b-4eb5-a634-230659c2dc15 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:56.984465 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54bd7e-5a1b-4eb5-a634-230659c2dc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:57.002356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3c54bd7e-5a1b-4eb5-a634-230659c2dc15 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:57.003591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3c54bd7e-5a1b-4eb5-a634-230659c2dc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:57.070571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1247/2556] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:57.080958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:57.082410 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:57.102305 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:57.145156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613f5e5-a8be-4a9c-b243-0d9809eb4276 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:57.148732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7db01a-b141-4a74-827e-8a88b75ab941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:57.149390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7db01a-b141-4a74-827e-8a88b75ab941 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:57.149681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7db01a-b141-4a74-827e-8a88b75ab941 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:57.149962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7db01a-b141-4a74-827e-8a88b75ab941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:57.159913 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7db01a-b141-4a74-827e-8a88b75ab941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:57.161402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7db01a-b141-4a74-827e-8a88b75ab941 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:57.171290 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1310/2557] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:02:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:02:57.466970 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54bd7e-5a1b-4eb5-a634-230659c2dc15 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:57.468436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3c54bd7e-5a1b-4eb5-a634-230659c2dc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:57.486877 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54bd7e-5a1b-4eb5-a634-230659c2dc15 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:57.491903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26335699-3163-482b-8cf3-0df4fd4e4eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:57.492805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26335699-3163-482b-8cf3-0df4fd4e4eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:57.493097 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26335699-3163-482b-8cf3-0df4fd4e4eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:57.493380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26335699-3163-482b-8cf3-0df4fd4e4eb7 admin admin] PATH_INFO: `/v3/projects/633f5ccb45f64ba9a716f1a572198866` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:57.495312 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26335699-3163-482b-8cf3-0df4fd4e4eb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=633f5ccb45f64ba9a716f1a572198866)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:57.497271 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26335699-3163-482b-8cf3-0df4fd4e4eb7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:57.553091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-26335699-3163-482b-8cf3-0df4fd4e4eb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '633f5ccb45f64ba9a716f1a572198866', 'request_id': 'req-26335699-3163-482b-8cf3-0df4fd4e4eb7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:57.562333 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-26335699-3163-482b-8cf3-0df4fd4e4eb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '633f5ccb45f64ba9a716f1a572198866', 'request_id': 'req-26335699-3163-482b-8cf3-0df4fd4e4eb7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:57.573259 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-26335699-3163-482b-8cf3-0df4fd4e4eb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '633f5ccb45f64ba9a716f1a572198866', 'request_id': 'req-26335699-3163-482b-8cf3-0df4fd4e4eb7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:57.586068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-26335699-3163-482b-8cf3-0df4fd4e4eb7 admin admin] The token cache is being invalidate because project 633f5ccb45f64ba9a716f1a572198866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:02:57.586503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-26335699-3163-482b-8cf3-0df4fd4e4eb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:57.587278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-26335699-3163-482b-8cf3-0df4fd4e4eb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:57.589256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1248/2558] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:02:57 2022] DELETE /identity/v3/projects/633f5ccb45f64ba9a716f1a572198866 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:02:57.594726 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c60a82-bbe8-4674-a1a3-188acbebb8ee None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:57.595027 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c60a82-bbe8-4674-a1a3-188acbebb8ee None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:57.595316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c60a82-bbe8-4674-a1a3-188acbebb8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:57.613950 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b8c60a82-bbe8-4674-a1a3-188acbebb8ee None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:57.615456 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b8c60a82-bbe8-4674-a1a3-188acbebb8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:57.716267 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1311/2559] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:57.724020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ec0d9-78bd-498c-b972-fb56c4b4b207 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:57.724236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ec0d9-78bd-498c-b972-fb56c4b4b207 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:57.724502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ec0d9-78bd-498c-b972-fb56c4b4b207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:57.751648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ec7ec0d9-78bd-498c-b972-fb56c4b4b207 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:57.753114 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ec7ec0d9-78bd-498c-b972-fb56c4b4b207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:57.786978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1249/2560] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:57.788937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c60a82-bbe8-4674-a1a3-188acbebb8ee None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:57.802841 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c60a82-bbe8-4674-a1a3-188acbebb8ee None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:57.818063 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ef738-b932-4e02-9fd0-22c1adceb1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:57.818840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ef738-b932-4e02-9fd0-22c1adceb1da admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:57.819337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ef738-b932-4e02-9fd0-22c1adceb1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:57.819705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ef738-b932-4e02-9fd0-22c1adceb1da admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:57.820526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ef738-b932-4e02-9fd0-22c1adceb1da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:02:57.820957 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ef738-b932-4e02-9fd0-22c1adceb1da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:57.822691 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1ef738-b932-4e02-9fd0-22c1adceb1da admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:57.848035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ec0d9-78bd-498c-b972-fb56c4b4b207 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:57.854683 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1312/2561] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:02:57.859187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ec0d9-78bd-498c-b972-fb56c4b4b207 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:57.865257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a37e48-fa95-4676-bd81-94c6fcf9225d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:57.866078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a37e48-fa95-4676-bd81-94c6fcf9225d admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:57.866384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a37e48-fa95-4676-bd81-94c6fcf9225d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:57.866729 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a37e48-fa95-4676-bd81-94c6fcf9225d admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:57.867999 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a37e48-fa95-4676-bd81-94c6fcf9225d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:57.870836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a37e48-fa95-4676-bd81-94c6fcf9225d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:57.912693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1250/2562] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:02:57 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:02:57.913158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ef738-b932-4e02-9fd0-22c1adceb1da admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:57.924870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1ef738-b932-4e02-9fd0-22c1adceb1da admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:57.997032 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1928e4c-e254-4989-a25d-e7223eb83b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:57.998108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1928e4c-e254-4989-a25d-e7223eb83b07 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:57.998645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1928e4c-e254-4989-a25d-e7223eb83b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:57.999185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1928e4c-e254-4989-a25d-e7223eb83b07 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:58.000539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1928e4c-e254-4989-a25d-e7223eb83b07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:58.002856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1928e4c-e254-4989-a25d-e7223eb83b07 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:58.030047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1313/2563] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:02:57 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:02:58.034402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a37e48-fa95-4676-bd81-94c6fcf9225d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:58.047734 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a37e48-fa95-4676-bd81-94c6fcf9225d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:58.061721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9f91a-6315-4785-9248-304a688050af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:58.063137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9f91a-6315-4785-9248-304a688050af admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:58.063137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9f91a-6315-4785-9248-304a688050af admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:58.063137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff9f91a-6315-4785-9248-304a688050af admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:58.064075 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9f91a-6315-4785-9248-304a688050af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:58.065875 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff9f91a-6315-4785-9248-304a688050af admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:58.081070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1251/2564] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:02:58.082336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1928e4c-e254-4989-a25d-e7223eb83b07 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:58.095354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1928e4c-e254-4989-a25d-e7223eb83b07 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:58.100390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a86754-1c8c-4281-923f-7785954f6532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:58.101339 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a86754-1c8c-4281-923f-7785954f6532 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:58.101632 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a86754-1c8c-4281-923f-7785954f6532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:58.102043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a86754-1c8c-4281-923f-7785954f6532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:58.102779 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a86754-1c8c-4281-923f-7785954f6532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:58.105131 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a86754-1c8c-4281-923f-7785954f6532 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:58.119676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1314/2565] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:02:58.120708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9f91a-6315-4785-9248-304a688050af admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:58.131489 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff9f91a-6315-4785-9248-304a688050af admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:58.137555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc2cf9-f60b-496a-9c4d-4432040bb59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:58.138624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc2cf9-f60b-496a-9c4d-4432040bb59b admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:58.138916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc2cf9-f60b-496a-9c4d-4432040bb59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:58.139537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc2cf9-f60b-496a-9c4d-4432040bb59b admin admin] PATH_INFO: `/v3/projects/2e6a7281e8274249a0313ad65a51266d/users/5c283611aa0743b9b06b9104b74ecb33/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:58.158121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc2cf9-f60b-496a-9c4d-4432040bb59b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6a7281e8274249a0313ad65a51266d, user_id=5c283611aa0743b9b06b9104b74ecb33, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:58.161659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc2cf9-f60b-496a-9c4d-4432040bb59b admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:58.172806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:58.196623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1252/2566] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:02:58 2022] PUT /identity/v3/projects/2e6a7281e8274249a0313ad65a51266d/users/5c283611aa0743b9b06b9104b74ecb33/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:02:58.201050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b436a4-b182-486c-a8dd-a17b388385c3 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:58.201346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b436a4-b182-486c-a8dd-a17b388385c3 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:58.201626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b436a4-b182-486c-a8dd-a17b388385c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:58.261529 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-93b436a4-b182-486c-a8dd-a17b388385c3 None None] MFA Rules not processed for user `5c283611aa0743b9b06b9104b74ecb33`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:58.262890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-93b436a4-b182-486c-a8dd-a17b388385c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:58.282160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:58.352528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1315/2567] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:02:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:58.358339 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc2cf9-f60b-496a-9c4d-4432040bb59b admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:58.436020 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc2cf9-f60b-496a-9c4d-4432040bb59b admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:58.441718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cebf7-e4f0-4ec7-bc2d-7106760bbd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:58.442522 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cebf7-e4f0-4ec7-bc2d-7106760bbd16 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:58.443002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cebf7-e4f0-4ec7-bc2d-7106760bbd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:58.443294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473cebf7-e4f0-4ec7-bc2d-7106760bbd16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:58.445039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cebf7-e4f0-4ec7-bc2d-7106760bbd16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:58.447891 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473cebf7-e4f0-4ec7-bc2d-7106760bbd16 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:58.479378 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1253/2568] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:02:58 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:02:58.483019 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b436a4-b182-486c-a8dd-a17b388385c3 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:58.494775 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b436a4-b182-486c-a8dd-a17b388385c3 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:58.554096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edc65a-9bd4-47b7-9f49-4fe1d9a9db1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:58.554736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edc65a-9bd4-47b7-9f49-4fe1d9a9db1b admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:58.555028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edc65a-9bd4-47b7-9f49-4fe1d9a9db1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:58.555312 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edc65a-9bd4-47b7-9f49-4fe1d9a9db1b admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:58.556005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edc65a-9bd4-47b7-9f49-4fe1d9a9db1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:58.558028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edc65a-9bd4-47b7-9f49-4fe1d9a9db1b admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:58.582085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1316/2569] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:02:58 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:02:58.586205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cebf7-e4f0-4ec7-bc2d-7106760bbd16 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:58.597667 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473cebf7-e4f0-4ec7-bc2d-7106760bbd16 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:58.602397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0f846-8861-4ab3-aa49-1a90a399ddb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:58.603061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0f846-8861-4ab3-aa49-1a90a399ddb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:58.603349 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0f846-8861-4ab3-aa49-1a90a399ddb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:58.603663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0f846-8861-4ab3-aa49-1a90a399ddb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:58.604397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0f846-8861-4ab3-aa49-1a90a399ddb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:58.606356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa0f846-8861-4ab3-aa49-1a90a399ddb2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:58.618062 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1254/2570] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:02:58.621888 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edc65a-9bd4-47b7-9f49-4fe1d9a9db1b admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:58.637724 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edc65a-9bd4-47b7-9f49-4fe1d9a9db1b admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:58.667096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af48b1-2627-4840-8b8d-1ffd33795d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:58.668363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af48b1-2627-4840-8b8d-1ffd33795d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:58.668863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af48b1-2627-4840-8b8d-1ffd33795d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:58.669440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af48b1-2627-4840-8b8d-1ffd33795d33 admin admin] PATH_INFO: `/v3/projects/b5485b0258bf42e1b40ee767976eab77/users/df147c0989af432d8c751a56a9e25f88/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:58.692666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af48b1-2627-4840-8b8d-1ffd33795d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5485b0258bf42e1b40ee767976eab77, user_id=df147c0989af432d8c751a56a9e25f88, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:58.695658 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af48b1-2627-4840-8b8d-1ffd33795d33 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:58.712869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1317/2571] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:02:58 2022] PUT /identity/v3/projects/b5485b0258bf42e1b40ee767976eab77/users/df147c0989af432d8c751a56a9e25f88/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:02:58.716085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0f846-8861-4ab3-aa49-1a90a399ddb2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:58.727294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0f846-8861-4ab3-aa49-1a90a399ddb2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:58.817444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439787ef-4e3c-4827-a220-a6b8e0965a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:58.818120 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439787ef-4e3c-4827-a220-a6b8e0965a76 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:58.818408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439787ef-4e3c-4827-a220-a6b8e0965a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:58.818694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439787ef-4e3c-4827-a220-a6b8e0965a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:58.819427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439787ef-4e3c-4827-a220-a6b8e0965a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:58.821076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439787ef-4e3c-4827-a220-a6b8e0965a76 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:58.836196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1255/2572] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:02:58.837720 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af48b1-2627-4840-8b8d-1ffd33795d33 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:58.849994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af48b1-2627-4840-8b8d-1ffd33795d33 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:58.855230 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8227e4-fe52-4388-93c3-66ed27ff3345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:58.856198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8227e4-fe52-4388-93c3-66ed27ff3345 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:58.856455 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8227e4-fe52-4388-93c3-66ed27ff3345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:58.856806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8227e4-fe52-4388-93c3-66ed27ff3345 admin admin] PATH_INFO: `/v3/projects/b5485b0258bf42e1b40ee767976eab77/users/df147c0989af432d8c751a56a9e25f88/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:58.859049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8227e4-fe52-4388-93c3-66ed27ff3345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5485b0258bf42e1b40ee767976eab77, user_id=df147c0989af432d8c751a56a9e25f88, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:58.861866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8227e4-fe52-4388-93c3-66ed27ff3345 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:58.874140 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1318/2573] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:02:58 2022] PUT /identity/v3/projects/b5485b0258bf42e1b40ee767976eab77/users/df147c0989af432d8c751a56a9e25f88/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:02:58.884325 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be93e4-1f4e-4ca2-9585-ddf92b6e1c0e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:58.884713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be93e4-1f4e-4ca2-9585-ddf92b6e1c0e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:58.885645 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85be93e4-1f4e-4ca2-9585-ddf92b6e1c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:58.917479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-85be93e4-1f4e-4ca2-9585-ddf92b6e1c0e None None] MFA Rules not processed for user `df147c0989af432d8c751a56a9e25f88`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:02:58.918724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-85be93e4-1f4e-4ca2-9585-ddf92b6e1c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:59.006819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:59.023389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1256/2574] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:02:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:02:59.034464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8227e4-fe52-4388-93c3-66ed27ff3345 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:59.036257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4e8227e4-fe52-4388-93c3-66ed27ff3345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:59.115200 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8227e4-fe52-4388-93c3-66ed27ff3345 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:59.124371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c03b0-bb3f-4f4d-8a82-f8b0a51099eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:59.125089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c03b0-bb3f-4f4d-8a82-f8b0a51099eb service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:59.125488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c03b0-bb3f-4f4d-8a82-f8b0a51099eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:59.125789 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553c03b0-bb3f-4f4d-8a82-f8b0a51099eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:59.137921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c03b0-bb3f-4f4d-8a82-f8b0a51099eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:59.139808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553c03b0-bb3f-4f4d-8a82-f8b0a51099eb service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:59.155789 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1319/2575] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:02:59 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:02:59.266151 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be93e4-1f4e-4ca2-9585-ddf92b6e1c0e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:59.282527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85be93e4-1f4e-4ca2-9585-ddf92b6e1c0e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:59.289655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278d5d1-d200-4073-bf3b-947df58ba7fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:59.291718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278d5d1-d200-4073-bf3b-947df58ba7fc service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:59.292240 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278d5d1-d200-4073-bf3b-947df58ba7fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:59.292823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e278d5d1-d200-4073-bf3b-947df58ba7fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:59.388918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278d5d1-d200-4073-bf3b-947df58ba7fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:59.395045 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e278d5d1-d200-4073-bf3b-947df58ba7fc service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:59.410120 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1257/2576] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:02:59 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:02:59.614520 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c03b0-bb3f-4f4d-8a82-f8b0a51099eb service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:02:59.617903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-553c03b0-bb3f-4f4d-8a82-f8b0a51099eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:02:59.633823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:02:59.710722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553c03b0-bb3f-4f4d-8a82-f8b0a51099eb service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:02:59.715483 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75128e49-2070-4815-afd8-4b2d2bd4f7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:02:59.716484 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75128e49-2070-4815-afd8-4b2d2bd4f7db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:02:59.716953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75128e49-2070-4815-afd8-4b2d2bd4f7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:02:59.717295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75128e49-2070-4815-afd8-4b2d2bd4f7db admin admin] PATH_INFO: `/v3/users/9cc891cc8172490394d2bfa3c8d1dcf8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:02:59.718534 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75128e49-2070-4815-afd8-4b2d2bd4f7db admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cc891cc8172490394d2bfa3c8d1dcf8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:02:59.721331 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75128e49-2070-4815-afd8-4b2d2bd4f7db admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:02:59.792294 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-75128e49-2070-4815-afd8-4b2d2bd4f7db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cc891cc8172490394d2bfa3c8d1dcf8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:59.793236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-75128e49-2070-4815-afd8-4b2d2bd4f7db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cc891cc8172490394d2bfa3c8d1dcf8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:59.794411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-75128e49-2070-4815-afd8-4b2d2bd4f7db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cc891cc8172490394d2bfa3c8d1dcf8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:59.809756 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-75128e49-2070-4815-afd8-4b2d2bd4f7db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cc891cc8172490394d2bfa3c8d1dcf8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:59.817916 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-75128e49-2070-4815-afd8-4b2d2bd4f7db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cc891cc8172490394d2bfa3c8d1dcf8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:02:59.835309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1320/2577] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:02:59 2022] DELETE /identity/v3/users/9cc891cc8172490394d2bfa3c8d1dcf8 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:00.042770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278d5d1-d200-4073-bf3b-947df58ba7fc service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:00.054478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e278d5d1-d200-4073-bf3b-947df58ba7fc service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:00.122495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:00.123365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:00.123654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:00.124009 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1 admin admin] PATH_INFO: `/v3/projects/b4f73dfee32d43b985e9192000eaed8c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:00.125199 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4f73dfee32d43b985e9192000eaed8c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:00.127116 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:00.175614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4f73dfee32d43b985e9192000eaed8c', 'request_id': 'req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:00.183519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4f73dfee32d43b985e9192000eaed8c', 'request_id': 'req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:00.189957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4f73dfee32d43b985e9192000eaed8c', 'request_id': 'req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:00.200172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1 admin admin] The token cache is being invalidate because project b4f73dfee32d43b985e9192000eaed8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:00.200480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:00.201136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c8b7bd8b-3e8b-4c73-89bf-d08b4c24b9f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:00.202662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1258/2578] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:00 2022] DELETE /identity/v3/projects/b4f73dfee32d43b985e9192000eaed8c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:00.207821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5ad74-e94e-45b4-b794-631a584b45ab None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:00.208303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5ad74-e94e-45b4-b794-631a584b45ab None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:00.208411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae5ad74-e94e-45b4-b794-631a584b45ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:00.226520 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bae5ad74-e94e-45b4-b794-631a584b45ab None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:00.227762 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bae5ad74-e94e-45b4-b794-631a584b45ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:00.290542 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1321/2579] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:00.298512 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec9fc3-9fef-484e-a1b3-ec61e55685e6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:00.299002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec9fc3-9fef-484e-a1b3-ec61e55685e6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:00.299106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec9fc3-9fef-484e-a1b3-ec61e55685e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:00.318343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-82ec9fc3-9fef-484e-a1b3-ec61e55685e6 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:00.319594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-82ec9fc3-9fef-484e-a1b3-ec61e55685e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:00.327586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1259/2580] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:00.331734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5ad74-e94e-45b4-b794-631a584b45ab None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:00.341985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae5ad74-e94e-45b4-b794-631a584b45ab None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:00.347110 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad12fd-c5f4-4376-8673-52e2cc15a78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:00.347765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad12fd-c5f4-4376-8673-52e2cc15a78f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:00.348053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad12fd-c5f4-4376-8673-52e2cc15a78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:00.348333 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad12fd-c5f4-4376-8673-52e2cc15a78f admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:00.349106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad12fd-c5f4-4376-8673-52e2cc15a78f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:03:00.349425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad12fd-c5f4-4376-8673-52e2cc15a78f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:00.351029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad12fd-c5f4-4376-8673-52e2cc15a78f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:00.364768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1322/2581] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:03:00.367926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec9fc3-9fef-484e-a1b3-ec61e55685e6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:00.378403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec9fc3-9fef-484e-a1b3-ec61e55685e6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:00.383403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9baa89-fb01-4cfb-9b59-cc8292bd00ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:00.384047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9baa89-fb01-4cfb-9b59-cc8292bd00ae admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:00.384332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9baa89-fb01-4cfb-9b59-cc8292bd00ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:00.384613 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9baa89-fb01-4cfb-9b59-cc8292bd00ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:00.385373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9baa89-fb01-4cfb-9b59-cc8292bd00ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:00.387289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9baa89-fb01-4cfb-9b59-cc8292bd00ae admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:00.401813 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:00.418359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1260/2582] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:00 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:00.423622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad12fd-c5f4-4376-8673-52e2cc15a78f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:00.434153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad12fd-c5f4-4376-8673-52e2cc15a78f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:00.445875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:00.502005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d3d77-c5cb-4470-8c07-68594cb7d318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:00.502684 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d3d77-c5cb-4470-8c07-68594cb7d318 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:00.503035 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d3d77-c5cb-4470-8c07-68594cb7d318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:00.503324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d3d77-c5cb-4470-8c07-68594cb7d318 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:00.504102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d3d77-c5cb-4470-8c07-68594cb7d318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:00.506173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3d3d77-c5cb-4470-8c07-68594cb7d318 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:00.533501 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1323/2583] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:00 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:00.537787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9baa89-fb01-4cfb-9b59-cc8292bd00ae admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:00.548708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9baa89-fb01-4cfb-9b59-cc8292bd00ae admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:00.553676 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a7cac-0a1a-447b-9928-84129451245c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:00.554417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a7cac-0a1a-447b-9928-84129451245c admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:00.554707 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a7cac-0a1a-447b-9928-84129451245c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:00.554986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a7cac-0a1a-447b-9928-84129451245c admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:00.555711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a7cac-0a1a-447b-9928-84129451245c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:00.557431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a7cac-0a1a-447b-9928-84129451245c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:00.571942 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1261/2584] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:00.573074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d3d77-c5cb-4470-8c07-68594cb7d318 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:00.583500 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d3d77-c5cb-4470-8c07-68594cb7d318 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:00.588814 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976b8c9-1855-4617-adab-7918b88445dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:00.589474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976b8c9-1855-4617-adab-7918b88445dd admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:00.589791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976b8c9-1855-4617-adab-7918b88445dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:00.590073 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976b8c9-1855-4617-adab-7918b88445dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:00.590800 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976b8c9-1855-4617-adab-7918b88445dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:00.592414 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976b8c9-1855-4617-adab-7918b88445dd admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:00.602596 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1324/2585] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:00.607347 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a7cac-0a1a-447b-9928-84129451245c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:00.617372 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a7cac-0a1a-447b-9928-84129451245c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:00.622418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa42f42-428c-4288-a38c-a3405c73e314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:00.623296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa42f42-428c-4288-a38c-a3405c73e314 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:00.623575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa42f42-428c-4288-a38c-a3405c73e314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:00.623848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa42f42-428c-4288-a38c-a3405c73e314 admin admin] PATH_INFO: `/v3/projects/860087fb6db04cd0b5ee816816a5a3fa/users/07d02953330044ce8d0c9d65591d0521/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:00.638983 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa42f42-428c-4288-a38c-a3405c73e314 admin admin] RBAC: Authorizing `identity:create_grant(project_id=860087fb6db04cd0b5ee816816a5a3fa, user_id=07d02953330044ce8d0c9d65591d0521, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:00.641717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa42f42-428c-4288-a38c-a3405c73e314 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:00.655345 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1262/2586] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:00 2022] PUT /identity/v3/projects/860087fb6db04cd0b5ee816816a5a3fa/users/07d02953330044ce8d0c9d65591d0521/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:00.665328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961377b-17dd-4165-a11f-f45093281364 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:00.665627 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961377b-17dd-4165-a11f-f45093281364 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:00.665933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961377b-17dd-4165-a11f-f45093281364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:00.692562 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a961377b-17dd-4165-a11f-f45093281364 None None] MFA Rules not processed for user `07d02953330044ce8d0c9d65591d0521`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:00.693820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a961377b-17dd-4165-a11f-f45093281364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:00.758786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1325/2587] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:00.770487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa42f42-428c-4288-a38c-a3405c73e314 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:00.772244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-efa42f42-428c-4288-a38c-a3405c73e314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:00.846489 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa42f42-428c-4288-a38c-a3405c73e314 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:00.852021 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba39229-fc1f-4807-8326-072ba1bd5f9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:00.852692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba39229-fc1f-4807-8326-072ba1bd5f9c service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:00.852990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba39229-fc1f-4807-8326-072ba1bd5f9c service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:00.853274 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba39229-fc1f-4807-8326-072ba1bd5f9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:00.861136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba39229-fc1f-4807-8326-072ba1bd5f9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:00.862756 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba39229-fc1f-4807-8326-072ba1bd5f9c service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:00.871578 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1263/2588] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:06.371077 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961377b-17dd-4165-a11f-f45093281364 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:06.372620 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a961377b-17dd-4165-a11f-f45093281364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:06.439400 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961377b-17dd-4165-a11f-f45093281364 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:06.443823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:06.444711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:06.445004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:06.445286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] PATH_INFO: `/v3/users/5c283611aa0743b9b06b9104b74ecb33` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:06.446472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c283611aa0743b9b06b9104b74ecb33)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:06.448401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:06.476539 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:06.495293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c283611aa0743b9b06b9104b74ecb33'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:06.496588 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c283611aa0743b9b06b9104b74ecb33'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:06.497230 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c283611aa0743b9b06b9104b74ecb33'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:06.508828 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c283611aa0743b9b06b9104b74ecb33'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:06.516913 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c283611aa0743b9b06b9104b74ecb33'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:06.530625 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1326/2589] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:06 2022] DELETE /identity/v3/users/5c283611aa0743b9b06b9104b74ecb33 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:06.535646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d512c0-5c50-47d5-b852-f6433bcb76c6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:06.536157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d512c0-5c50-47d5-b852-f6433bcb76c6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:06.536270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d512c0-5c50-47d5-b852-f6433bcb76c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:06.554112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-54d512c0-5c50-47d5-b852-f6433bcb76c6 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:06.555411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-54d512c0-5c50-47d5-b852-f6433bcb76c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:06.618702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1264/2590] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:06.628388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:06.629870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:06.686927 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45a787-c3a5-4894-b623-37bf93161117 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:06.690579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18a2c4-efd6-48ba-b802-0d979434647a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:06.691158 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18a2c4-efd6-48ba-b802-0d979434647a service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:06.691446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18a2c4-efd6-48ba-b802-0d979434647a service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:06.691722 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc18a2c4-efd6-48ba-b802-0d979434647a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:06.701053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18a2c4-efd6-48ba-b802-0d979434647a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:06.702468 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc18a2c4-efd6-48ba-b802-0d979434647a service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:06.711692 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1327/2591] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:06.986791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d512c0-5c50-47d5-b852-f6433bcb76c6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:06.988920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-54d512c0-5c50-47d5-b852-f6433bcb76c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:07.006258 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d512c0-5c50-47d5-b852-f6433bcb76c6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:07.017350 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:07.018760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:07.019072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:07.019896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] PATH_INFO: `/v3/projects/2e6a7281e8274249a0313ad65a51266d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:07.020973 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6a7281e8274249a0313ad65a51266d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:07.022923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:07.075737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:INSERT=2,keystone:SELECT=95,keystone:DELETE=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:07.084294 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6a7281e8274249a0313ad65a51266d', 'request_id': 'req-f2835561-cca7-43aa-a5e6-3b258879ac47'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.093633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6a7281e8274249a0313ad65a51266d', 'request_id': 'req-f2835561-cca7-43aa-a5e6-3b258879ac47'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.100064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6a7281e8274249a0313ad65a51266d', 'request_id': 'req-f2835561-cca7-43aa-a5e6-3b258879ac47'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.110495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] The token cache is being invalidate because project 2e6a7281e8274249a0313ad65a51266d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:07.110788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.111450 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.113034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1265/2592] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:06 2022] DELETE /identity/v3/projects/2e6a7281e8274249a0313ad65a51266d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:07.117437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18a2c4-efd6-48ba-b802-0d979434647a service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:07.127748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc18a2c4-efd6-48ba-b802-0d979434647a service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:07.186207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8debb2-0895-4d2f-aeb6-59325622fc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:07.187093 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8debb2-0895-4d2f-aeb6-59325622fc1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:07.187384 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8debb2-0895-4d2f-aeb6-59325622fc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:07.187682 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8debb2-0895-4d2f-aeb6-59325622fc1c admin admin] PATH_INFO: `/v3/users/df147c0989af432d8c751a56a9e25f88` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:07.189004 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8debb2-0895-4d2f-aeb6-59325622fc1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=df147c0989af432d8c751a56a9e25f88)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:07.191083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8debb2-0895-4d2f-aeb6-59325622fc1c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:07.244977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8a8debb2-0895-4d2f-aeb6-59325622fc1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df147c0989af432d8c751a56a9e25f88'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.246184 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8a8debb2-0895-4d2f-aeb6-59325622fc1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df147c0989af432d8c751a56a9e25f88'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.246825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8a8debb2-0895-4d2f-aeb6-59325622fc1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df147c0989af432d8c751a56a9e25f88'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.248026 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:07.262879 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8a8debb2-0895-4d2f-aeb6-59325622fc1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df147c0989af432d8c751a56a9e25f88'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.269805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8a8debb2-0895-4d2f-aeb6-59325622fc1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df147c0989af432d8c751a56a9e25f88'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.282157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1328/2593] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:07 2022] DELETE /identity/v3/users/df147c0989af432d8c751a56a9e25f88 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:07.549835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:07.551571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:07.565024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2835561-cca7-43aa-a5e6-3b258879ac47 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:07.624560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cccab69-717c-429d-ad40-66bdb2e7900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:07.625462 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccab69-717c-429d-ad40-66bdb2e7900e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:07.625752 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccab69-717c-429d-ad40-66bdb2e7900e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:07.626035 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cccab69-717c-429d-ad40-66bdb2e7900e admin admin] PATH_INFO: `/v3/projects/b5485b0258bf42e1b40ee767976eab77` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:07.627236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cccab69-717c-429d-ad40-66bdb2e7900e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5485b0258bf42e1b40ee767976eab77)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:07.629208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cccab69-717c-429d-ad40-66bdb2e7900e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:07.675183 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2cccab69-717c-429d-ad40-66bdb2e7900e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5485b0258bf42e1b40ee767976eab77', 'request_id': 'req-2cccab69-717c-429d-ad40-66bdb2e7900e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.684016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2cccab69-717c-429d-ad40-66bdb2e7900e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5485b0258bf42e1b40ee767976eab77', 'request_id': 'req-2cccab69-717c-429d-ad40-66bdb2e7900e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.691040 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2cccab69-717c-429d-ad40-66bdb2e7900e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5485b0258bf42e1b40ee767976eab77', 'request_id': 'req-2cccab69-717c-429d-ad40-66bdb2e7900e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.701986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2cccab69-717c-429d-ad40-66bdb2e7900e admin admin] The token cache is being invalidate because project b5485b0258bf42e1b40ee767976eab77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:07.702304 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2cccab69-717c-429d-ad40-66bdb2e7900e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.702996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2cccab69-717c-429d-ad40-66bdb2e7900e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:07.704648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1266/2594] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:07 2022] DELETE /identity/v3/projects/b5485b0258bf42e1b40ee767976eab77 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:07.710249 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2c560-0740-46f2-b1e8-b2d49732371b None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:07.710714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2c560-0740-46f2-b1e8-b2d49732371b None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:07.710873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a2c560-0740-46f2-b1e8-b2d49732371b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:07.729518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-71a2c560-0740-46f2-b1e8-b2d49732371b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:07.731371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-71a2c560-0740-46f2-b1e8-b2d49732371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:07.788959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1329/2595] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:07.797104 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768ff7f-fca8-4b84-8043-0a7e3a222408 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:07.797403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768ff7f-fca8-4b84-8043-0a7e3a222408 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:07.797684 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768ff7f-fca8-4b84-8043-0a7e3a222408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:07.816728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c768ff7f-fca8-4b84-8043-0a7e3a222408 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:07.817976 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c768ff7f-fca8-4b84-8043-0a7e3a222408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:07.826209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1267/2596] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:07.830693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2c560-0740-46f2-b1e8-b2d49732371b None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:07.841560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a2c560-0740-46f2-b1e8-b2d49732371b None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:07.846299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea405e0f-4024-4754-b092-e118e137e9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:07.846957 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea405e0f-4024-4754-b092-e118e137e9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:07.847252 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea405e0f-4024-4754-b092-e118e137e9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:07.847532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea405e0f-4024-4754-b092-e118e137e9d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:07.848260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea405e0f-4024-4754-b092-e118e137e9d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:03:07.848575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea405e0f-4024-4754-b092-e118e137e9d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:07.850200 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea405e0f-4024-4754-b092-e118e137e9d5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:07.862425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1330/2597] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:03:07.866876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768ff7f-fca8-4b84-8043-0a7e3a222408 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:07.876363 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768ff7f-fca8-4b84-8043-0a7e3a222408 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:07.881083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657f01e-b7a5-481f-9068-1d70bcab2ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:07.881730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657f01e-b7a5-481f-9068-1d70bcab2ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:07.882019 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657f01e-b7a5-481f-9068-1d70bcab2ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:07.882298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657f01e-b7a5-481f-9068-1d70bcab2ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:07.882996 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657f01e-b7a5-481f-9068-1d70bcab2ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:07.885058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657f01e-b7a5-481f-9068-1d70bcab2ca1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:07.910563 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1268/2598] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:07 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:07.914815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea405e0f-4024-4754-b092-e118e137e9d5 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:07.925870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea405e0f-4024-4754-b092-e118e137e9d5 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:07.981545 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a692d-7a59-4834-818b-359216265538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:07.982215 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a692d-7a59-4834-818b-359216265538 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:07.982508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a692d-7a59-4834-818b-359216265538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:07.982792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a692d-7a59-4834-818b-359216265538 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:07.983492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a692d-7a59-4834-818b-359216265538 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:07.985490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a692d-7a59-4834-818b-359216265538 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:08.005334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1331/2599] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:07 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:08.009830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657f01e-b7a5-481f-9068-1d70bcab2ca1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:08.020150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657f01e-b7a5-481f-9068-1d70bcab2ca1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:08.024867 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615a22d-c86d-40f7-b8dd-9dd40fe721f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:08.025522 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615a22d-c86d-40f7-b8dd-9dd40fe721f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:08.025806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615a22d-c86d-40f7-b8dd-9dd40fe721f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:08.026086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615a22d-c86d-40f7-b8dd-9dd40fe721f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:08.027313 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615a22d-c86d-40f7-b8dd-9dd40fe721f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:08.028936 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615a22d-c86d-40f7-b8dd-9dd40fe721f7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:08.039335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1269/2600] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:03:08.043732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a692d-7a59-4834-818b-359216265538 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:08.053984 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a692d-7a59-4834-818b-359216265538 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:08.058685 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598d3a1-a935-430c-b1e6-69a32126dc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:08.059337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598d3a1-a935-430c-b1e6-69a32126dc62 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:08.059626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598d3a1-a935-430c-b1e6-69a32126dc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:08.059908 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598d3a1-a935-430c-b1e6-69a32126dc62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:08.060633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598d3a1-a935-430c-b1e6-69a32126dc62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:08.062265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598d3a1-a935-430c-b1e6-69a32126dc62 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:08.072882 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1332/2601] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:08.077168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615a22d-c86d-40f7-b8dd-9dd40fe721f7 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:08.088924 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615a22d-c86d-40f7-b8dd-9dd40fe721f7 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:08.094277 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c7f3e-51b9-4615-bd60-f815aa4f1ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:08.095170 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c7f3e-51b9-4615-bd60-f815aa4f1ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:08.095461 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c7f3e-51b9-4615-bd60-f815aa4f1ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:08.095753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c7f3e-51b9-4615-bd60-f815aa4f1ce1 admin admin] PATH_INFO: `/v3/projects/27f07ee8e41a42a2bc434d4d16b48b0f/users/d709e152862c4e67828f7504e7d13438/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:08.111555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c7f3e-51b9-4615-bd60-f815aa4f1ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f07ee8e41a42a2bc434d4d16b48b0f, user_id=d709e152862c4e67828f7504e7d13438, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:08.116307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c7f3e-51b9-4615-bd60-f815aa4f1ce1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:08.125296 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:08.132502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1270/2602] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:08 2022] PUT /identity/v3/projects/27f07ee8e41a42a2bc434d4d16b48b0f/users/d709e152862c4e67828f7504e7d13438/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:08.144038 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23d96c-3938-436f-9c25-43072f393d5f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:08.144336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23d96c-3938-436f-9c25-43072f393d5f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:08.144663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23d96c-3938-436f-9c25-43072f393d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:08.173851 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-3c23d96c-3938-436f-9c25-43072f393d5f None None] MFA Rules not processed for user `d709e152862c4e67828f7504e7d13438`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:08.174466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3c23d96c-3938-436f-9c25-43072f393d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:08.195843 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:08.219389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c7f3e-51b9-4615-bd60-f815aa4f1ce1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:08.219389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d41c7f3e-51b9-4615-bd60-f815aa4f1ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:08.262171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1333/2603] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:08.269947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23d96c-3938-436f-9c25-43072f393d5f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:08.271447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-3c23d96c-3938-436f-9c25-43072f393d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:08.293917 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c7f3e-51b9-4615-bd60-f815aa4f1ce1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:08.298619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2a59e-a2fb-46a2-b0f3-5dda1dd991a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:08.299536 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2a59e-a2fb-46a2-b0f3-5dda1dd991a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:08.299810 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2a59e-a2fb-46a2-b0f3-5dda1dd991a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:08.300094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2a59e-a2fb-46a2-b0f3-5dda1dd991a8 admin admin] PATH_INFO: `/v3/users/07d02953330044ce8d0c9d65591d0521` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:08.301929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2a59e-a2fb-46a2-b0f3-5dda1dd991a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07d02953330044ce8d0c9d65591d0521)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:08.303576 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a2a59e-a2fb-46a2-b0f3-5dda1dd991a8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:08.346314 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23d96c-3938-436f-9c25-43072f393d5f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:08.351642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ce600-d0fa-440a-a9ef-ff30beed7764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:08.352541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ce600-d0fa-440a-a9ef-ff30beed7764 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:08.352887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ce600-d0fa-440a-a9ef-ff30beed7764 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:08.353170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ce600-d0fa-440a-a9ef-ff30beed7764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:08.363527 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ce600-d0fa-440a-a9ef-ff30beed7764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:08.365356 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ce600-d0fa-440a-a9ef-ff30beed7764 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:08.372079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09a2a59e-a2fb-46a2-b0f3-5dda1dd991a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07d02953330044ce8d0c9d65591d0521'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:08.373158 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09a2a59e-a2fb-46a2-b0f3-5dda1dd991a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07d02953330044ce8d0c9d65591d0521'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:08.374086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09a2a59e-a2fb-46a2-b0f3-5dda1dd991a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07d02953330044ce8d0c9d65591d0521'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:08.383522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1334/2604] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:03:08.394456 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09a2a59e-a2fb-46a2-b0f3-5dda1dd991a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07d02953330044ce8d0c9d65591d0521'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:08.406655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-09a2a59e-a2fb-46a2-b0f3-5dda1dd991a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07d02953330044ce8d0c9d65591d0521'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:08.407507 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ce600-d0fa-440a-a9ef-ff30beed7764 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:08.417968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1271/2605] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:08 2022] DELETE /identity/v3/users/07d02953330044ce8d0c9d65591d0521 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:08.421144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ce600-d0fa-440a-a9ef-ff30beed7764 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:08.422297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0711d-08ae-4ce4-a3f1-55b32b34aa7b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:08.422624 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0711d-08ae-4ce4-a3f1-55b32b34aa7b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:08.422995 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d0711d-08ae-4ce4-a3f1-55b32b34aa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:08.440872 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-09d0711d-08ae-4ce4-a3f1-55b32b34aa7b None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:08.442130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-09d0711d-08ae-4ce4-a3f1-55b32b34aa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:08.487496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e0839-6683-4cda-bb1c-32d2d3994eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:08.488410 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e0839-6683-4cda-bb1c-32d2d3994eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:08.488705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e0839-6683-4cda-bb1c-32d2d3994eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:08.489214 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e0839-6683-4cda-bb1c-32d2d3994eea admin admin] PATH_INFO: `/v3/users/d709e152862c4e67828f7504e7d13438` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:08.490570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e0839-6683-4cda-bb1c-32d2d3994eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d709e152862c4e67828f7504e7d13438)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:08.492533 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e0839-6683-4cda-bb1c-32d2d3994eea admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:08.508374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1272/2606] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:08.519557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0711d-08ae-4ce4-a3f1-55b32b34aa7b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:08.521138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-09d0711d-08ae-4ce4-a3f1-55b32b34aa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:08.538717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:08.557810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-812e0839-6683-4cda-bb1c-32d2d3994eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd709e152862c4e67828f7504e7d13438'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:08.558936 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-812e0839-6683-4cda-bb1c-32d2d3994eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd709e152862c4e67828f7504e7d13438'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:08.559666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-812e0839-6683-4cda-bb1c-32d2d3994eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd709e152862c4e67828f7504e7d13438'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:08.571864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-812e0839-6683-4cda-bb1c-32d2d3994eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd709e152862c4e67828f7504e7d13438'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:08.573055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:08.580618 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-812e0839-6683-4cda-bb1c-32d2d3994eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd709e152862c4e67828f7504e7d13438'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:08.583498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d0711d-08ae-4ce4-a3f1-55b32b34aa7b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:08.587831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b9a1e-a120-46be-bb5a-aabe6b70008c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:08.588477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b9a1e-a120-46be-bb5a-aabe6b70008c service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:08.588845 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b9a1e-a120-46be-bb5a-aabe6b70008c service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:08.589182 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b9a1e-a120-46be-bb5a-aabe6b70008c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:08.591289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6e8b9a1e-a120-46be-bb5a-aabe6b70008c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:08.596801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1335/2607] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:08 2022] DELETE /identity/v3/users/d709e152862c4e67828f7504e7d13438 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:03:08.601194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc03d6-ccaa-4630-aa43-5d9c966bc27c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:08.601497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc03d6-ccaa-4630-aa43-5d9c966bc27c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:08.601786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc03d6-ccaa-4630-aa43-5d9c966bc27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:08.604071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b9a1e-a120-46be-bb5a-aabe6b70008c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:08.605596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b9a1e-a120-46be-bb5a-aabe6b70008c service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:08.617135 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1273/2608] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:03:08.618954 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-43fc03d6-ccaa-4630-aa43-5d9c966bc27c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:08.620189 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-43fc03d6-ccaa-4630-aa43-5d9c966bc27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:08.684346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1336/2609] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:08.694903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b9a1e-a120-46be-bb5a-aabe6b70008c service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:08.696581 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-6e8b9a1e-a120-46be-bb5a-aabe6b70008c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:08.708327 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b9a1e-a120-46be-bb5a-aabe6b70008c service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:08.758936 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428ad48-51ca-4df7-9839-d3fddc16b178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:08.759592 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428ad48-51ca-4df7-9839-d3fddc16b178 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:08.759881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428ad48-51ca-4df7-9839-d3fddc16b178 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:08.760163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428ad48-51ca-4df7-9839-d3fddc16b178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:08.773853 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428ad48-51ca-4df7-9839-d3fddc16b178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:08.775326 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428ad48-51ca-4df7-9839-d3fddc16b178 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:08.786053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1274/2610] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:03:08.991715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc03d6-ccaa-4630-aa43-5d9c966bc27c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:08.993260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-43fc03d6-ccaa-4630-aa43-5d9c966bc27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:09.006157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc03d6-ccaa-4630-aa43-5d9c966bc27c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.010765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.013574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.013874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.014180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] PATH_INFO: `/v3/projects/860087fb6db04cd0b5ee816816a5a3fa` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.016978 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=860087fb6db04cd0b5ee816816a5a3fa)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.019060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.082662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860087fb6db04cd0b5ee816816a5a3fa', 'request_id': 'req-84b3c39f-71e1-482b-989e-d3434a9bae93'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:09.092082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860087fb6db04cd0b5ee816816a5a3fa', 'request_id': 'req-84b3c39f-71e1-482b-989e-d3434a9bae93'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:09.098017 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860087fb6db04cd0b5ee816816a5a3fa', 'request_id': 'req-84b3c39f-71e1-482b-989e-d3434a9bae93'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:09.109869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] The token cache is being invalidate because project 860087fb6db04cd0b5ee816816a5a3fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:09.110163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:09.110869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:09.112429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1337/2611] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:08 2022] DELETE /identity/v3/projects/860087fb6db04cd0b5ee816816a5a3fa => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:09.117351 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f7e46-1f81-4c51-9932-c029c63ab418 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.117643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f7e46-1f81-4c51-9932-c029c63ab418 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.117927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f7e46-1f81-4c51-9932-c029c63ab418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.138478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e08f7e46-1f81-4c51-9932-c029c63ab418 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:09.139717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e08f7e46-1f81-4c51-9932-c029c63ab418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:09.174274 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.176136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:09.225239 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1275/2612] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:09.235593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191348c-216b-4afb-80aa-fc21fc611dbb None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.236031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191348c-216b-4afb-80aa-fc21fc611dbb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.236460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b191348c-216b-4afb-80aa-fc21fc611dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.256102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3c39f-71e1-482b-989e-d3434a9bae93 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.258810 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b191348c-216b-4afb-80aa-fc21fc611dbb None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:09.260043 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b191348c-216b-4afb-80aa-fc21fc611dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:09.260830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d096e3b-75f8-4f09-beed-03b6d88b6d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.262283 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d096e3b-75f8-4f09-beed-03b6d88b6d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.262570 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d096e3b-75f8-4f09-beed-03b6d88b6d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.262844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d096e3b-75f8-4f09-beed-03b6d88b6d09 admin admin] PATH_INFO: `/v3/projects/27f07ee8e41a42a2bc434d4d16b48b0f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.264046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d096e3b-75f8-4f09-beed-03b6d88b6d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f07ee8e41a42a2bc434d4d16b48b0f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.265969 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d096e3b-75f8-4f09-beed-03b6d88b6d09 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.269910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1276/2613] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:09.274041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191348c-216b-4afb-80aa-fc21fc611dbb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.274494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:09.283491 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b191348c-216b-4afb-80aa-fc21fc611dbb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.288279 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8a555-003f-4b44-a8e3-234d108c6564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.288964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8a555-003f-4b44-a8e3-234d108c6564 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.289251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8a555-003f-4b44-a8e3-234d108c6564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.289530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8a555-003f-4b44-a8e3-234d108c6564 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.290251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8a555-003f-4b44-a8e3-234d108c6564 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:03:09.290560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8a555-003f-4b44-a8e3-234d108c6564 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.292153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8a555-003f-4b44-a8e3-234d108c6564 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.305183 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1277/2614] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 12:03:09.309932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8a555-003f-4b44-a8e3-234d108c6564 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.315907 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8d096e3b-75f8-4f09-beed-03b6d88b6d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f07ee8e41a42a2bc434d4d16b48b0f', 'request_id': 'req-8d096e3b-75f8-4f09-beed-03b6d88b6d09'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:09.319958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8a555-003f-4b44-a8e3-234d108c6564 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.325993 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8d096e3b-75f8-4f09-beed-03b6d88b6d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f07ee8e41a42a2bc434d4d16b48b0f', 'request_id': 'req-8d096e3b-75f8-4f09-beed-03b6d88b6d09'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:09.332449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8d096e3b-75f8-4f09-beed-03b6d88b6d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f07ee8e41a42a2bc434d4d16b48b0f', 'request_id': 'req-8d096e3b-75f8-4f09-beed-03b6d88b6d09'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:09.345392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8d096e3b-75f8-4f09-beed-03b6d88b6d09 admin admin] The token cache is being invalidate because project 27f07ee8e41a42a2bc434d4d16b48b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:09.346127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8d096e3b-75f8-4f09-beed-03b6d88b6d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:09.346559 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-8d096e3b-75f8-4f09-beed-03b6d88b6d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:09.348342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1338/2615] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:09 2022] DELETE /identity/v3/projects/27f07ee8e41a42a2bc434d4d16b48b0f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:03:09.352832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ba036-9949-4d09-afb1-fb32cedf68fe None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.353146 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ba036-9949-4d09-afb1-fb32cedf68fe None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.354011 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ba036-9949-4d09-afb1-fb32cedf68fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.363362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:09.372430 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-919ba036-9949-4d09-afb1-fb32cedf68fe None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:09.373801 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-919ba036-9949-4d09-afb1-fb32cedf68fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:09.384402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db277b0-e340-4c5f-b4c8-bf3f87724759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.385095 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db277b0-e340-4c5f-b4c8-bf3f87724759 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.385382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db277b0-e340-4c5f-b4c8-bf3f87724759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.385670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db277b0-e340-4c5f-b4c8-bf3f87724759 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.386428 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db277b0-e340-4c5f-b4c8-bf3f87724759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.388434 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db277b0-e340-4c5f-b4c8-bf3f87724759 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.413646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1278/2616] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:09 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:03:09.418315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db277b0-e340-4c5f-b4c8-bf3f87724759 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.420087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5db277b0-e340-4c5f-b4c8-bf3f87724759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:09.438076 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1339/2617] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:09.445626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287b934-3a67-4a99-8d1d-eb9eee1b74fe None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.445926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287b934-3a67-4a99-8d1d-eb9eee1b74fe None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.446218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287b934-3a67-4a99-8d1d-eb9eee1b74fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.464395 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-0287b934-3a67-4a99-8d1d-eb9eee1b74fe None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:09.465658 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0287b934-3a67-4a99-8d1d-eb9eee1b74fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:09.487181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db277b0-e340-4c5f-b4c8-bf3f87724759 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.491884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40fde0-b28d-4598-8c86-5cab55c69886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.492548 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40fde0-b28d-4598-8c86-5cab55c69886 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.492998 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40fde0-b28d-4598-8c86-5cab55c69886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.493284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40fde0-b28d-4598-8c86-5cab55c69886 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.493971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40fde0-b28d-4598-8c86-5cab55c69886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.496570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e40fde0-b28d-4598-8c86-5cab55c69886 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.516577 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1279/2618] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:09 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:03:09.520496 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40fde0-b28d-4598-8c86-5cab55c69886 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.530678 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40fde0-b28d-4598-8c86-5cab55c69886 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.533432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1340/2619] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:09.535628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e6999-d562-48ce-be1f-c3c40fe09618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.536286 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e6999-d562-48ce-be1f-c3c40fe09618 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.536577 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e6999-d562-48ce-be1f-c3c40fe09618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.536885 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e6999-d562-48ce-be1f-c3c40fe09618 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.537381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287b934-3a67-4a99-8d1d-eb9eee1b74fe None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.537612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e6999-d562-48ce-be1f-c3c40fe09618 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.539198 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e6999-d562-48ce-be1f-c3c40fe09618 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.547800 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287b934-3a67-4a99-8d1d-eb9eee1b74fe None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.550629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1280/2620] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:03:09.552819 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f32ef7-1349-4ae1-b453-4af1d692f014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.553797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f32ef7-1349-4ae1-b453-4af1d692f014 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.553998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f32ef7-1349-4ae1-b453-4af1d692f014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.554206 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e6999-d562-48ce-be1f-c3c40fe09618 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.554312 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f32ef7-1349-4ae1-b453-4af1d692f014 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.555018 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f32ef7-1349-4ae1-b453-4af1d692f014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:03:09.555303 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f32ef7-1349-4ae1-b453-4af1d692f014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.557017 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f32ef7-1349-4ae1-b453-4af1d692f014 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.564532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e6999-d562-48ce-be1f-c3c40fe09618 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.569370 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc09a01-c0d7-4b3b-9f4c-287d6ffa46f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.569594 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1341/2621] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:03:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:03:09.569968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc09a01-c0d7-4b3b-9f4c-287d6ffa46f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.570257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc09a01-c0d7-4b3b-9f4c-287d6ffa46f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.570539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc09a01-c0d7-4b3b-9f4c-287d6ffa46f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.571265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc09a01-c0d7-4b3b-9f4c-287d6ffa46f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.572904 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc09a01-c0d7-4b3b-9f4c-287d6ffa46f4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.573918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f32ef7-1349-4ae1-b453-4af1d692f014 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.586421 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1281/2622] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:03:09.586945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f32ef7-1349-4ae1-b453-4af1d692f014 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.587286 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc09a01-c0d7-4b3b-9f4c-287d6ffa46f4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.589626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182e295-1622-413c-8827-4fb66a6ff240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.590464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182e295-1622-413c-8827-4fb66a6ff240 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.590759 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182e295-1622-413c-8827-4fb66a6ff240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.591041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182e295-1622-413c-8827-4fb66a6ff240 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.591856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b182e295-1622-413c-8827-4fb66a6ff240 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.593869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b182e295-1622-413c-8827-4fb66a6ff240 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.597129 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc09a01-c0d7-4b3b-9f4c-287d6ffa46f4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.601706 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00535070-cf62-48cc-958f-511512651822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.602570 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00535070-cf62-48cc-958f-511512651822 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.602857 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00535070-cf62-48cc-958f-511512651822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.603143 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00535070-cf62-48cc-958f-511512651822 admin admin] PATH_INFO: `/v3/projects/1de0b887db9343869d02b96a0969b5e2/users/7f780a4caf4b4fc9a607fd2fa07d6ebb/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.618991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00535070-cf62-48cc-958f-511512651822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de0b887db9343869d02b96a0969b5e2, user_id=7f780a4caf4b4fc9a607fd2fa07d6ebb, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.623874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1342/2623] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:09 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:03:09.624029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00535070-cf62-48cc-958f-511512651822 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.625120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182e295-1622-413c-8827-4fb66a6ff240 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.630366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:09.638070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1282/2624] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:09 2022] PUT /identity/v3/projects/1de0b887db9343869d02b96a0969b5e2/users/7f780a4caf4b4fc9a607fd2fa07d6ebb/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:03:09.648991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d40eb-b1ad-4714-a3b7-506f6b8b7c1e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.649307 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d40eb-b1ad-4714-a3b7-506f6b8b7c1e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.649596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25d40eb-b1ad-4714-a3b7-506f6b8b7c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.678341 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e25d40eb-b1ad-4714-a3b7-506f6b8b7c1e None None] MFA Rules not processed for user `7f780a4caf4b4fc9a607fd2fa07d6ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:09.679619 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e25d40eb-b1ad-4714-a3b7-506f6b8b7c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:09.694709 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182e295-1622-413c-8827-4fb66a6ff240 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.742064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:09.747222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1283/2625] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:09 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:09.752048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d40eb-b1ad-4714-a3b7-506f6b8b7c1e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.753703 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82932cb-9af8-47a7-a9a3-0609d3e93973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.754502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82932cb-9af8-47a7-a9a3-0609d3e93973 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.754793 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82932cb-9af8-47a7-a9a3-0609d3e93973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.755072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82932cb-9af8-47a7-a9a3-0609d3e93973 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.755765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82932cb-9af8-47a7-a9a3-0609d3e93973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.757764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82932cb-9af8-47a7-a9a3-0609d3e93973 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.763232 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25d40eb-b1ad-4714-a3b7-506f6b8b7c1e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.768209 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7236e25-9d28-40fd-b095-d583cee1e07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.768876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7236e25-9d28-40fd-b095-d583cee1e07f admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.769261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7236e25-9d28-40fd-b095-d583cee1e07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.769663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7236e25-9d28-40fd-b095-d583cee1e07f admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.770500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7236e25-9d28-40fd-b095-d583cee1e07f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.772541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7236e25-9d28-40fd-b095-d583cee1e07f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.778587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1343/2626] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:09 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:03:09.782710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82932cb-9af8-47a7-a9a3-0609d3e93973 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.793824 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82932cb-9af8-47a7-a9a3-0609d3e93973 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.798598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adddc9-4859-40e9-bcba-42c3844d04e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.799257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adddc9-4859-40e9-bcba-42c3844d04e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.799544 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adddc9-4859-40e9-bcba-42c3844d04e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.799827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adddc9-4859-40e9-bcba-42c3844d04e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.800596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1284/2627] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:09 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:03:09.803968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adddc9-4859-40e9-bcba-42c3844d04e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.803968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78adddc9-4859-40e9-bcba-42c3844d04e8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.804770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7236e25-9d28-40fd-b095-d583cee1e07f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.815379 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1344/2628] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:03:09.816363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adddc9-4859-40e9-bcba-42c3844d04e8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.826633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78adddc9-4859-40e9-bcba-42c3844d04e8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.875390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7236e25-9d28-40fd-b095-d583cee1e07f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.880435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcb26b-3dce-40e2-b45b-daf74069d051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.881145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcb26b-3dce-40e2-b45b-daf74069d051 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.881436 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcb26b-3dce-40e2-b45b-daf74069d051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.881719 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcb26b-3dce-40e2-b45b-daf74069d051 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.882539 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcb26b-3dce-40e2-b45b-daf74069d051 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.884502 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcb26b-3dce-40e2-b45b-daf74069d051 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.885107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139fc0b-f7b4-4725-987d-1c6c47ec2b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.885777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139fc0b-f7b4-4725-987d-1c6c47ec2b02 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.886067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139fc0b-f7b4-4725-987d-1c6c47ec2b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.886376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b139fc0b-f7b4-4725-987d-1c6c47ec2b02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.887162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139fc0b-f7b4-4725-987d-1c6c47ec2b02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.888777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b139fc0b-f7b4-4725-987d-1c6c47ec2b02 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.899022 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1345/2629] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:09.903697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139fc0b-f7b4-4725-987d-1c6c47ec2b02 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.905893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1285/2630] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:09 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:03:09.909675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcb26b-3dce-40e2-b45b-daf74069d051 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.916827 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b139fc0b-f7b4-4725-987d-1c6c47ec2b02 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.919908 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcb26b-3dce-40e2-b45b-daf74069d051 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.922295 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d596483-0aa8-4ee5-86fc-85596dee6d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.923178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d596483-0aa8-4ee5-86fc-85596dee6d46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.923469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d596483-0aa8-4ee5-86fc-85596dee6d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.923750 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d596483-0aa8-4ee5-86fc-85596dee6d46 admin admin] PATH_INFO: `/v3/projects/2d1d02e4165a4631b89f87a19cb7ae74/users/06b0af3b57164ca0aef15be4d0bc09e7/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.924638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d905311-8df3-4fa1-9360-0c7eaf61591f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:09.925681 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d905311-8df3-4fa1-9360-0c7eaf61591f admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.925887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d905311-8df3-4fa1-9360-0c7eaf61591f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.926172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d905311-8df3-4fa1-9360-0c7eaf61591f admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.926900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d905311-8df3-4fa1-9360-0c7eaf61591f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.928505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d905311-8df3-4fa1-9360-0c7eaf61591f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.939440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d596483-0aa8-4ee5-86fc-85596dee6d46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1d02e4165a4631b89f87a19cb7ae74, user_id=06b0af3b57164ca0aef15be4d0bc09e7, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:09.939884 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1286/2631] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:03:09.942144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d596483-0aa8-4ee5-86fc-85596dee6d46 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:09.944034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d905311-8df3-4fa1-9360-0c7eaf61591f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:09.952175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:09.955186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d905311-8df3-4fa1-9360-0c7eaf61591f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:09.958154 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1346/2632] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:09 2022] PUT /identity/v3/projects/2d1d02e4165a4631b89f87a19cb7ae74/users/06b0af3b57164ca0aef15be4d0bc09e7/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:03:09.968730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8100dd1-4d2c-49f5-872c-7945d20e90d8 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:09.969346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8100dd1-4d2c-49f5-872c-7945d20e90d8 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:09.970074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8100dd1-4d2c-49f5-872c-7945d20e90d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:09.997860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b8100dd1-4d2c-49f5-872c-7945d20e90d8 None None] MFA Rules not processed for user `06b0af3b57164ca0aef15be4d0bc09e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:09.999116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b8100dd1-4d2c-49f5-872c-7945d20e90d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:10.013589 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52df0fe-420b-4bf7-858b-46b7984a0329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:10.014507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52df0fe-420b-4bf7-858b-46b7984a0329 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.014794 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52df0fe-420b-4bf7-858b-46b7984a0329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.015074 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52df0fe-420b-4bf7-858b-46b7984a0329 admin admin] PATH_INFO: `/v3/projects/728d3c065c7548b99518e0e3e6f50a1b/users/496414a779be4c93b50a0038a06860b7/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.030154 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52df0fe-420b-4bf7-858b-46b7984a0329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728d3c065c7548b99518e0e3e6f50a1b, user_id=496414a779be4c93b50a0038a06860b7, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:10.033388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52df0fe-420b-4bf7-858b-46b7984a0329 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:10.047164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1287/2633] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:09 2022] PUT /identity/v3/projects/728d3c065c7548b99518e0e3e6f50a1b/users/496414a779be4c93b50a0038a06860b7/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:10.051397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52df0fe-420b-4bf7-858b-46b7984a0329 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:10.064776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1347/2634] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:10.069206 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8100dd1-4d2c-49f5-872c-7945d20e90d8 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:10.080046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8100dd1-4d2c-49f5-872c-7945d20e90d8 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:10.094499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:10.116473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52df0fe-420b-4bf7-858b-46b7984a0329 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:10.125627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627ef5d-32b0-4c62-bea5-b6078a036089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:10.126341 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627ef5d-32b0-4c62-bea5-b6078a036089 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.126628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627ef5d-32b0-4c62-bea5-b6078a036089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.126911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627ef5d-32b0-4c62-bea5-b6078a036089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.127646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627ef5d-32b0-4c62-bea5-b6078a036089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:10.129302 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b627ef5d-32b0-4c62-bea5-b6078a036089 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:10.140941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1288/2635] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:03:10.142084 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b86ce-315e-4e17-9d84-e73fe5b18b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:10.142805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b86ce-315e-4e17-9d84-e73fe5b18b6d admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.143100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b86ce-315e-4e17-9d84-e73fe5b18b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.143385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b86ce-315e-4e17-9d84-e73fe5b18b6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.144120 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b86ce-315e-4e17-9d84-e73fe5b18b6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:10.145183 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627ef5d-32b0-4c62-bea5-b6078a036089 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:10.146185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b86ce-315e-4e17-9d84-e73fe5b18b6d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:10.161523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627ef5d-32b0-4c62-bea5-b6078a036089 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:10.166985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14362a7c-40eb-4c01-8076-ef2639bf003f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:10.168251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14362a7c-40eb-4c01-8076-ef2639bf003f admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.168651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14362a7c-40eb-4c01-8076-ef2639bf003f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.169046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14362a7c-40eb-4c01-8076-ef2639bf003f admin admin] PATH_INFO: `/v3/projects/728d3c065c7548b99518e0e3e6f50a1b/users/496414a779be4c93b50a0038a06860b7/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.171787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14362a7c-40eb-4c01-8076-ef2639bf003f admin admin] RBAC: Authorizing `identity:create_grant(project_id=728d3c065c7548b99518e0e3e6f50a1b, user_id=496414a779be4c93b50a0038a06860b7, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:10.175053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14362a7c-40eb-4c01-8076-ef2639bf003f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:10.187259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1348/2636] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:10 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:03:10.187460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1289/2637] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:10 2022] PUT /identity/v3/projects/728d3c065c7548b99518e0e3e6f50a1b/users/496414a779be4c93b50a0038a06860b7/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:10.193783 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b86ce-315e-4e17-9d84-e73fe5b18b6d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:10.196728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0aaff-4d72-49c3-9b78-6dc3a5c17766 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.197032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0aaff-4d72-49c3-9b78-6dc3a5c17766 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.197373 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0aaff-4d72-49c3-9b78-6dc3a5c17766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.227107 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-a7f0aaff-4d72-49c3-9b78-6dc3a5c17766 None None] MFA Rules not processed for user `496414a779be4c93b50a0038a06860b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:10.228470 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a7f0aaff-4d72-49c3-9b78-6dc3a5c17766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:10.263385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b86ce-315e-4e17-9d84-e73fe5b18b6d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:10.268096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d8e5b-2df8-451a-ade5-d8c26f5e4fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:10.268796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d8e5b-2df8-451a-ade5-d8c26f5e4fcf admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.269081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d8e5b-2df8-451a-ade5-d8c26f5e4fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.269355 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d8e5b-2df8-451a-ade5-d8c26f5e4fcf admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.270038 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d8e5b-2df8-451a-ade5-d8c26f5e4fcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:10.271944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d8e5b-2df8-451a-ade5-d8c26f5e4fcf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:10.292596 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1349/2638] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:10 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:10.296263 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d8e5b-2df8-451a-ade5-d8c26f5e4fcf admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:10.302659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:10.306944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d8e5b-2df8-451a-ade5-d8c26f5e4fcf admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:10.311725 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1290/2639] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:10 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:10.312008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19d7e7-e414-498e-84bb-f6020429428c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:10.313244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19d7e7-e414-498e-84bb-f6020429428c admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.313629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19d7e7-e414-498e-84bb-f6020429428c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.314261 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19d7e7-e414-498e-84bb-f6020429428c admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.314512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19d7e7-e414-498e-84bb-f6020429428c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:10.316861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19d7e7-e414-498e-84bb-f6020429428c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:10.319042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0aaff-4d72-49c3-9b78-6dc3a5c17766 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:10.320441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a7f0aaff-4d72-49c3-9b78-6dc3a5c17766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:10.327518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1350/2640] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:03:10.331739 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19d7e7-e414-498e-84bb-f6020429428c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:10.341865 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19d7e7-e414-498e-84bb-f6020429428c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:10.346606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cbcca-7ef4-492a-8a3f-dc11c54d21d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:10.347474 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cbcca-7ef4-492a-8a3f-dc11c54d21d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.347760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cbcca-7ef4-492a-8a3f-dc11c54d21d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.348039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cbcca-7ef4-492a-8a3f-dc11c54d21d7 admin admin] PATH_INFO: `/v3/projects/6c458589408342b3867e6520b6a1e291/users/de76c3644e0a4b52a4afd78d67e3c66a/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.362734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cbcca-7ef4-492a-8a3f-dc11c54d21d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c458589408342b3867e6520b6a1e291, user_id=de76c3644e0a4b52a4afd78d67e3c66a, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:10.365432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195cbcca-7ef4-492a-8a3f-dc11c54d21d7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:10.378404 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1351/2641] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:10 2022] PUT /identity/v3/projects/6c458589408342b3867e6520b6a1e291/users/de76c3644e0a4b52a4afd78d67e3c66a/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:03:10.381604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0aaff-4d72-49c3-9b78-6dc3a5c17766 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:10.382454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cbcca-7ef4-492a-8a3f-dc11c54d21d7 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:10.431730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644a4f3-3165-4150-ad10-f27632c45663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:10.432374 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644a4f3-3165-4150-ad10-f27632c45663 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.432771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644a4f3-3165-4150-ad10-f27632c45663 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.432987 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644a4f3-3165-4150-ad10-f27632c45663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.444260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cbcca-7ef4-492a-8a3f-dc11c54d21d7 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:10.447519 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:10.450301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3d209-a0f4-4d4f-a9ef-610ae9deb43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:10.450985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3d209-a0f4-4d4f-a9ef-610ae9deb43f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.451673 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3d209-a0f4-4d4f-a9ef-610ae9deb43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.451673 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f3d209-a0f4-4d4f-a9ef-610ae9deb43f admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.452488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3d209-a0f4-4d4f-a9ef-610ae9deb43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:10.454665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f3d209-a0f4-4d4f-a9ef-610ae9deb43f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:10.466489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1352/2642] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:03:10.470791 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3d209-a0f4-4d4f-a9ef-610ae9deb43f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:10.480955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f3d209-a0f4-4d4f-a9ef-610ae9deb43f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:10.485760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863da55b-af39-4bfa-a648-81886b7cc5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:10.486641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863da55b-af39-4bfa-a648-81886b7cc5a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.486930 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863da55b-af39-4bfa-a648-81886b7cc5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.487221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863da55b-af39-4bfa-a648-81886b7cc5a2 admin admin] PATH_INFO: `/v3/projects/6c458589408342b3867e6520b6a1e291/users/de76c3644e0a4b52a4afd78d67e3c66a/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.489278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863da55b-af39-4bfa-a648-81886b7cc5a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c458589408342b3867e6520b6a1e291, user_id=de76c3644e0a4b52a4afd78d67e3c66a, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:10.491944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863da55b-af39-4bfa-a648-81886b7cc5a2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:10.503272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1353/2643] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:10 2022] PUT /identity/v3/projects/6c458589408342b3867e6520b6a1e291/users/de76c3644e0a4b52a4afd78d67e3c66a/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:10.510772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644a4f3-3165-4150-ad10-f27632c45663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:10.513050 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644a4f3-3165-4150-ad10-f27632c45663 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:10.515383 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407fd2f-c72f-4de1-96bf-92cc7bec03ed None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.515772 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407fd2f-c72f-4de1-96bf-92cc7bec03ed None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.516067 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407fd2f-c72f-4de1-96bf-92cc7bec03ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.523988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1291/2644] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:10 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:10.543586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-4407fd2f-c72f-4de1-96bf-92cc7bec03ed None None] MFA Rules not processed for user `de76c3644e0a4b52a4afd78d67e3c66a`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:10.544860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4407fd2f-c72f-4de1-96bf-92cc7bec03ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:10.626132 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1354/2645] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:10.637432 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644a4f3-3165-4150-ad10-f27632c45663 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:10.646457 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644a4f3-3165-4150-ad10-f27632c45663 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:10.703921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2268ef-cebb-4435-b6b5-0c84893938c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:10.704584 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2268ef-cebb-4435-b6b5-0c84893938c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:10.705102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2268ef-cebb-4435-b6b5-0c84893938c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:10.705284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2268ef-cebb-4435-b6b5-0c84893938c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:10.772767 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2268ef-cebb-4435-b6b5-0c84893938c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:10.774435 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2268ef-cebb-4435-b6b5-0c84893938c5 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:10.784511 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1292/2646] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:10.949602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407fd2f-c72f-4de1-96bf-92cc7bec03ed None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:10.997343 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:11.023914 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407fd2f-c72f-4de1-96bf-92cc7bec03ed None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:11.032871 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:11.034432 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:11.034639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:11.034926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] PATH_INFO: `/v3/users/7f780a4caf4b4fc9a607fd2fa07d6ebb` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:11.036472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f780a4caf4b4fc9a607fd2fa07d6ebb)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:11.038869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:11.099196 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f780a4caf4b4fc9a607fd2fa07d6ebb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.099987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f780a4caf4b4fc9a607fd2fa07d6ebb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.100752 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f780a4caf4b4fc9a607fd2fa07d6ebb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.113724 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f780a4caf4b4fc9a607fd2fa07d6ebb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.121467 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f780a4caf4b4fc9a607fd2fa07d6ebb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.134666 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1355/2647] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:10 2022] DELETE /identity/v3/users/7f780a4caf4b4fc9a607fd2fa07d6ebb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:11.139206 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97b770-d9bc-4861-89af-5e135c4df8f0 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:11.139507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97b770-d9bc-4861-89af-5e135c4df8f0 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:11.139817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f97b770-d9bc-4861-89af-5e135c4df8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:11.156881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3f97b770-d9bc-4861-89af-5e135c4df8f0 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:11.158166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3f97b770-d9bc-4861-89af-5e135c4df8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:11.221001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1293/2648] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:11.231428 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:11.233100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:11.290197 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d50297-3348-42c9-9aaf-3a27e8d5c970 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:11.293875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f291e12-5c76-4abc-b57f-89d8e3157250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:11.294520 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f291e12-5c76-4abc-b57f-89d8e3157250 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:11.294808 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f291e12-5c76-4abc-b57f-89d8e3157250 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:11.295091 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f291e12-5c76-4abc-b57f-89d8e3157250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:11.304369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f291e12-5c76-4abc-b57f-89d8e3157250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:11.305820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f291e12-5c76-4abc-b57f-89d8e3157250 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:11.315717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1356/2649] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:11.588609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97b770-d9bc-4861-89af-5e135c4df8f0 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:11.590459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3f97b770-d9bc-4861-89af-5e135c4df8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:11.602848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f97b770-d9bc-4861-89af-5e135c4df8f0 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:11.608208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:11.609108 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:11.609397 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:11.609678 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] PATH_INFO: `/v3/projects/1de0b887db9343869d02b96a0969b5e2` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:11.610906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1de0b887db9343869d02b96a0969b5e2)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:11.612896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:11.614549 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:11.661241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1de0b887db9343869d02b96a0969b5e2', 'request_id': 'req-2af38b46-3725-4be3-be6c-09e22d8528d7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.671024 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1de0b887db9343869d02b96a0969b5e2', 'request_id': 'req-2af38b46-3725-4be3-be6c-09e22d8528d7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.677818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1de0b887db9343869d02b96a0969b5e2', 'request_id': 'req-2af38b46-3725-4be3-be6c-09e22d8528d7'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.688512 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] The token cache is being invalidate because project 1de0b887db9343869d02b96a0969b5e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:11.688839 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.689638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.691346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1294/2650] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:11 2022] DELETE /identity/v3/projects/1de0b887db9343869d02b96a0969b5e2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:11.695473 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f291e12-5c76-4abc-b57f-89d8e3157250 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:11.705220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f291e12-5c76-4abc-b57f-89d8e3157250 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:11.761764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ff663-82b9-4ee7-849d-a126748b8f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:11.762626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ff663-82b9-4ee7-849d-a126748b8f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:11.762959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ff663-82b9-4ee7-849d-a126748b8f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:11.763218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ff663-82b9-4ee7-849d-a126748b8f21 admin admin] PATH_INFO: `/v3/users/496414a779be4c93b50a0038a06860b7` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:11.764417 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ff663-82b9-4ee7-849d-a126748b8f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=496414a779be4c93b50a0038a06860b7)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:11.766380 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ff663-82b9-4ee7-849d-a126748b8f21 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:11.767876 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:11.815043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e45ff663-82b9-4ee7-849d-a126748b8f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '496414a779be4c93b50a0038a06860b7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.815707 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e45ff663-82b9-4ee7-849d-a126748b8f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '496414a779be4c93b50a0038a06860b7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.816322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e45ff663-82b9-4ee7-849d-a126748b8f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '496414a779be4c93b50a0038a06860b7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.827941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e45ff663-82b9-4ee7-849d-a126748b8f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '496414a779be4c93b50a0038a06860b7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.834659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e45ff663-82b9-4ee7-849d-a126748b8f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '496414a779be4c93b50a0038a06860b7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:11.847714 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1357/2651] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:11 2022] DELETE /identity/v3/users/496414a779be4c93b50a0038a06860b7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:12.126906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:12.128883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:12.156411 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af38b46-3725-4be3-be6c-09e22d8528d7 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:12.226876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1dae6-0b46-457f-8eff-65843f33ba81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:12.227776 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1dae6-0b46-457f-8eff-65843f33ba81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:12.228087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1dae6-0b46-457f-8eff-65843f33ba81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:12.229142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1dae6-0b46-457f-8eff-65843f33ba81 admin admin] PATH_INFO: `/v3/projects/728d3c065c7548b99518e0e3e6f50a1b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:12.230452 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1dae6-0b46-457f-8eff-65843f33ba81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=728d3c065c7548b99518e0e3e6f50a1b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:12.233337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1dae6-0b46-457f-8eff-65843f33ba81 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:12.292208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3ad1dae6-0b46-457f-8eff-65843f33ba81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728d3c065c7548b99518e0e3e6f50a1b', 'request_id': 'req-3ad1dae6-0b46-457f-8eff-65843f33ba81'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:12.302777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3ad1dae6-0b46-457f-8eff-65843f33ba81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728d3c065c7548b99518e0e3e6f50a1b', 'request_id': 'req-3ad1dae6-0b46-457f-8eff-65843f33ba81'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:12.315355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3ad1dae6-0b46-457f-8eff-65843f33ba81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728d3c065c7548b99518e0e3e6f50a1b', 'request_id': 'req-3ad1dae6-0b46-457f-8eff-65843f33ba81'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:12.329949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3ad1dae6-0b46-457f-8eff-65843f33ba81 admin admin] The token cache is being invalidate because project 728d3c065c7548b99518e0e3e6f50a1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:12.330382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3ad1dae6-0b46-457f-8eff-65843f33ba81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:12.331192 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-3ad1dae6-0b46-457f-8eff-65843f33ba81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:12.333594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1295/2652] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:12 2022] DELETE /identity/v3/projects/728d3c065c7548b99518e0e3e6f50a1b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:12.340160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730559e-de78-4955-ba02-e7c338b7434a None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:12.340758 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730559e-de78-4955-ba02-e7c338b7434a None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:12.340758 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730559e-de78-4955-ba02-e7c338b7434a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:12.370764 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-7730559e-de78-4955-ba02-e7c338b7434a None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:12.372160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-7730559e-de78-4955-ba02-e7c338b7434a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:12.446531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1358/2653] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:12.460087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a293b-22e2-456c-94b3-1c07fcd3d499 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:12.460625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a293b-22e2-456c-94b3-1c07fcd3d499 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:12.461546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8a293b-22e2-456c-94b3-1c07fcd3d499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:12.484202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-ae8a293b-22e2-456c-94b3-1c07fcd3d499 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:12.485526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ae8a293b-22e2-456c-94b3-1c07fcd3d499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:12.496605 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1296/2654] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:12.504803 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730559e-de78-4955-ba02-e7c338b7434a None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:12.516489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730559e-de78-4955-ba02-e7c338b7434a None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:12.523956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e84ed-02af-4075-83d5-9cd9fca71f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:12.528940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e84ed-02af-4075-83d5-9cd9fca71f3f admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:12.528940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e84ed-02af-4075-83d5-9cd9fca71f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:12.528940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e84ed-02af-4075-83d5-9cd9fca71f3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:12.528940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e84ed-02af-4075-83d5-9cd9fca71f3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:03:12.528940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e84ed-02af-4075-83d5-9cd9fca71f3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:12.528940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e84ed-02af-4075-83d5-9cd9fca71f3f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:12.542498 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1359/2655] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:03:12.547565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a293b-22e2-456c-94b3-1c07fcd3d499 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:12.559288 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8a293b-22e2-456c-94b3-1c07fcd3d499 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:12.564276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fddec8-c7ba-4b64-bbec-579f553bc721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:12.565017 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fddec8-c7ba-4b64-bbec-579f553bc721 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:12.565297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fddec8-c7ba-4b64-bbec-579f553bc721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:12.565531 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fddec8-c7ba-4b64-bbec-579f553bc721 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:12.566286 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fddec8-c7ba-4b64-bbec-579f553bc721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:12.568136 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fddec8-c7ba-4b64-bbec-579f553bc721 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:12.585661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:12.609953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1297/2656] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:12 2022] POST /identity/v3/projects => generated 365 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:03:12.611175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e84ed-02af-4075-83d5-9cd9fca71f3f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:12.621370 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e84ed-02af-4075-83d5-9cd9fca71f3f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:12.685780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35213058-a578-4446-81ee-f31f2c6c9809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:12.686490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35213058-a578-4446-81ee-f31f2c6c9809 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:12.686780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35213058-a578-4446-81ee-f31f2c6c9809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:12.687065 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35213058-a578-4446-81ee-f31f2c6c9809 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:12.687768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35213058-a578-4446-81ee-f31f2c6c9809 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:12.689732 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35213058-a578-4446-81ee-f31f2c6c9809 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:12.701493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:12.718043 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1360/2657] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:12 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:12.724126 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fddec8-c7ba-4b64-bbec-579f553bc721 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:12.734578 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fddec8-c7ba-4b64-bbec-579f553bc721 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:12.739166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffa355-82ce-4741-b549-61eb4d35a1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:12.739832 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffa355-82ce-4741-b549-61eb4d35a1be admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:12.740119 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffa355-82ce-4741-b549-61eb4d35a1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:12.740396 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ffa355-82ce-4741-b549-61eb4d35a1be admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:12.741168 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffa355-82ce-4741-b549-61eb4d35a1be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:12.742762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ffa355-82ce-4741-b549-61eb4d35a1be admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:12.755893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1298/2658] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:12.760890 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35213058-a578-4446-81ee-f31f2c6c9809 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:12.774127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35213058-a578-4446-81ee-f31f2c6c9809 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:12.778903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e003df-aaf8-48f8-b621-48914509de54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:12.779569 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e003df-aaf8-48f8-b621-48914509de54 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:12.779998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e003df-aaf8-48f8-b621-48914509de54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:12.780285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e003df-aaf8-48f8-b621-48914509de54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:12.781141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e003df-aaf8-48f8-b621-48914509de54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:12.782992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e003df-aaf8-48f8-b621-48914509de54 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:12.796247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1361/2659] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:12.800949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffa355-82ce-4741-b549-61eb4d35a1be admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:12.814299 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ffa355-82ce-4741-b549-61eb4d35a1be admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:12.820034 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc13d7d-0a32-47ae-852d-c6c9cb790f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:12.820958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc13d7d-0a32-47ae-852d-c6c9cb790f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:12.821250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc13d7d-0a32-47ae-852d-c6c9cb790f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:12.821533 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc13d7d-0a32-47ae-852d-c6c9cb790f09 admin admin] PATH_INFO: `/v3/projects/6b9ac089fd32463b8fca23536dbf2944/users/46a28397a8ea4f0bbfdbbb6e1057e29c/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:12.838376 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc13d7d-0a32-47ae-852d-c6c9cb790f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b9ac089fd32463b8fca23536dbf2944, user_id=46a28397a8ea4f0bbfdbbb6e1057e29c, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:12.842400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc13d7d-0a32-47ae-852d-c6c9cb790f09 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:12.856380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1299/2660] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:12 2022] PUT /identity/v3/projects/6b9ac089fd32463b8fca23536dbf2944/users/46a28397a8ea4f0bbfdbbb6e1057e29c/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:12.869809 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42694b1-2393-4778-80a4-8a9a0eb546eb None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:12.870105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42694b1-2393-4778-80a4-8a9a0eb546eb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:12.870992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42694b1-2393-4778-80a4-8a9a0eb546eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:12.900199 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a42694b1-2393-4778-80a4-8a9a0eb546eb None None] MFA Rules not processed for user `46a28397a8ea4f0bbfdbbb6e1057e29c`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:12.901469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a42694b1-2393-4778-80a4-8a9a0eb546eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:12.965925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1362/2661] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:12.970478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc13d7d-0a32-47ae-852d-c6c9cb790f09 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:13.031477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc13d7d-0a32-47ae-852d-c6c9cb790f09 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:13.036103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b7d6f-bd4e-4f8f-a19a-e6eed410a2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:13.036783 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b7d6f-bd4e-4f8f-a19a-e6eed410a2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:13.037078 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b7d6f-bd4e-4f8f-a19a-e6eed410a2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:13.037344 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b7d6f-bd4e-4f8f-a19a-e6eed410a2e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:13.038042 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b7d6f-bd4e-4f8f-a19a-e6eed410a2e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:13.039932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b7d6f-bd4e-4f8f-a19a-e6eed410a2e0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:13.070847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1300/2662] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:12 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:13.075620 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42694b1-2393-4778-80a4-8a9a0eb546eb None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:13.086466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42694b1-2393-4778-80a4-8a9a0eb546eb None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:13.141626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5d48c-e5fc-4fbd-ad9f-bdc1010fcef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:13.142284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5d48c-e5fc-4fbd-ad9f-bdc1010fcef8 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:13.142556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5d48c-e5fc-4fbd-ad9f-bdc1010fcef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:13.142834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5d48c-e5fc-4fbd-ad9f-bdc1010fcef8 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:13.143568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5d48c-e5fc-4fbd-ad9f-bdc1010fcef8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:13.145513 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5d48c-e5fc-4fbd-ad9f-bdc1010fcef8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:13.165994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1363/2663] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:13 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:13.170646 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b7d6f-bd4e-4f8f-a19a-e6eed410a2e0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:13.181225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b7d6f-bd4e-4f8f-a19a-e6eed410a2e0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:13.186565 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb503a-da51-4d55-8800-c66a5fd8f0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:13.187222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb503a-da51-4d55-8800-c66a5fd8f0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:13.187511 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb503a-da51-4d55-8800-c66a5fd8f0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:13.187791 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb503a-da51-4d55-8800-c66a5fd8f0c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:13.188528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb503a-da51-4d55-8800-c66a5fd8f0c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:13.190157 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb503a-da51-4d55-8800-c66a5fd8f0c8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:13.201364 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1301/2664] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:13.205891 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5d48c-e5fc-4fbd-ad9f-bdc1010fcef8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:13.215962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5d48c-e5fc-4fbd-ad9f-bdc1010fcef8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:13.220592 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cd5c4-91a2-4437-b315-defce1cd8e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:13.221508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cd5c4-91a2-4437-b315-defce1cd8e4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:13.221795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cd5c4-91a2-4437-b315-defce1cd8e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:13.222072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cd5c4-91a2-4437-b315-defce1cd8e4e admin admin] PATH_INFO: `/v3/projects/0bd73e82b5524fad995417c033d112cc/users/f6596e1dabe7428bb41b123f10106a5c/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:13.238036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cd5c4-91a2-4437-b315-defce1cd8e4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd73e82b5524fad995417c033d112cc, user_id=f6596e1dabe7428bb41b123f10106a5c, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:13.240494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cd5c4-91a2-4437-b315-defce1cd8e4e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:13.254736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1364/2665] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:13 2022] PUT /identity/v3/projects/0bd73e82b5524fad995417c033d112cc/users/f6596e1dabe7428bb41b123f10106a5c/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:03:13.258848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb503a-da51-4d55-8800-c66a5fd8f0c8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:13.268863 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb503a-da51-4d55-8800-c66a5fd8f0c8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:13.324899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c1cc9-8709-447c-a713-69208820b305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:13.325474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c1cc9-8709-447c-a713-69208820b305 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:13.325757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c1cc9-8709-447c-a713-69208820b305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:13.326030 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c1cc9-8709-447c-a713-69208820b305 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:13.326742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c1cc9-8709-447c-a713-69208820b305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:13.328316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c1cc9-8709-447c-a713-69208820b305 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:13.341458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1302/2666] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:13.345743 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cd5c4-91a2-4437-b315-defce1cd8e4e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:13.356466 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cd5c4-91a2-4437-b315-defce1cd8e4e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:13.361032 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183266d3-516d-4d4e-ad72-8a1fb3e15324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:13.361898 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183266d3-516d-4d4e-ad72-8a1fb3e15324 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:13.362183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183266d3-516d-4d4e-ad72-8a1fb3e15324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:13.362457 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183266d3-516d-4d4e-ad72-8a1fb3e15324 admin admin] PATH_INFO: `/v3/projects/0bd73e82b5524fad995417c033d112cc/users/f6596e1dabe7428bb41b123f10106a5c/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:13.364382 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183266d3-516d-4d4e-ad72-8a1fb3e15324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd73e82b5524fad995417c033d112cc, user_id=f6596e1dabe7428bb41b123f10106a5c, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:13.367037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183266d3-516d-4d4e-ad72-8a1fb3e15324 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:13.378178 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1365/2667] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:13 2022] PUT /identity/v3/projects/0bd73e82b5524fad995417c033d112cc/users/f6596e1dabe7428bb41b123f10106a5c/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:13.389250 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e17a0-4d00-40cc-9df0-ba40ff801c25 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:13.389617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e17a0-4d00-40cc-9df0-ba40ff801c25 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:13.389730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183266d3-516d-4d4e-ad72-8a1fb3e15324 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:13.390141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e17a0-4d00-40cc-9df0-ba40ff801c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:13.391440 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-183266d3-516d-4d4e-ad72-8a1fb3e15324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:13.417812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-457e17a0-4d00-40cc-9df0-ba40ff801c25 None None] MFA Rules not processed for user `f6596e1dabe7428bb41b123f10106a5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:13.419044 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-457e17a0-4d00-40cc-9df0-ba40ff801c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:13.423604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:13.433555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:13.461938 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183266d3-516d-4d4e-ad72-8a1fb3e15324 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:13.467909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be53e63-4225-442d-abf6-13a9524a72c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:13.468974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be53e63-4225-442d-abf6-13a9524a72c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:13.469381 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be53e63-4225-442d-abf6-13a9524a72c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:13.469777 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be53e63-4225-442d-abf6-13a9524a72c7 admin admin] PATH_INFO: `/v3/users/06b0af3b57164ca0aef15be4d0bc09e7` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:13.471636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be53e63-4225-442d-abf6-13a9524a72c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06b0af3b57164ca0aef15be4d0bc09e7)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:13.475157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be53e63-4225-442d-abf6-13a9524a72c7 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:13.502920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1303/2668] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:13 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:13.514784 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e17a0-4d00-40cc-9df0-ba40ff801c25 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:13.516822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-457e17a0-4d00-40cc-9df0-ba40ff801c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:13.535257 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4be53e63-4225-442d-abf6-13a9524a72c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06b0af3b57164ca0aef15be4d0bc09e7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:13.535951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4be53e63-4225-442d-abf6-13a9524a72c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06b0af3b57164ca0aef15be4d0bc09e7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:13.536610 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4be53e63-4225-442d-abf6-13a9524a72c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06b0af3b57164ca0aef15be4d0bc09e7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:13.550838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4be53e63-4225-442d-abf6-13a9524a72c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06b0af3b57164ca0aef15be4d0bc09e7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:13.558700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4be53e63-4225-442d-abf6-13a9524a72c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06b0af3b57164ca0aef15be4d0bc09e7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:13.572095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1366/2669] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:13 2022] DELETE /identity/v3/users/06b0af3b57164ca0aef15be4d0bc09e7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:03:13.575920 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e17a0-4d00-40cc-9df0-ba40ff801c25 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:13.576530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d584b67-de1c-452b-9395-b31fca48484e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:13.576851 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d584b67-de1c-452b-9395-b31fca48484e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:13.577138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d584b67-de1c-452b-9395-b31fca48484e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:13.595131 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-1d584b67-de1c-452b-9395-b31fca48484e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:13.596388 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1d584b67-de1c-452b-9395-b31fca48484e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:13.627106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cf4b6-83b2-4983-9478-e4f38477dc30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:13.627749 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cf4b6-83b2-4983-9478-e4f38477dc30 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:13.628032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cf4b6-83b2-4983-9478-e4f38477dc30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:13.628545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cf4b6-83b2-4983-9478-e4f38477dc30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:13.630407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-344cf4b6-83b2-4983-9478-e4f38477dc30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:13.656229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1367/2670] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:13.665979 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d584b67-de1c-452b-9395-b31fca48484e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:13.668245 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-1d584b67-de1c-452b-9395-b31fca48484e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:13.696941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344cf4b6-83b2-4983-9478-e4f38477dc30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:13.698384 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344cf4b6-83b2-4983-9478-e4f38477dc30 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:13.708362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1304/2671] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:13 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:13.722064 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:13.728399 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d584b67-de1c-452b-9395-b31fca48484e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:13.735317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3cd0f-3077-433b-b232-41c6b573b078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:13.736061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3cd0f-3077-433b-b232-41c6b573b078 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:13.737054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3cd0f-3077-433b-b232-41c6b573b078 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:13.737373 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3cd0f-3077-433b-b232-41c6b573b078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:13.747552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3cd0f-3077-433b-b232-41c6b573b078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:13.749190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3cd0f-3077-433b-b232-41c6b573b078 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:13.759336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1368/2672] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:03:13.782948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cf4b6-83b2-4983-9478-e4f38477dc30 service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:13.784453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-344cf4b6-83b2-4983-9478-e4f38477dc30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:13.795909 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cf4b6-83b2-4983-9478-e4f38477dc30 service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:13.799719 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d33ede-a27a-47c0-9a4f-99a25a02aecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:13.800766 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d33ede-a27a-47c0-9a4f-99a25a02aecd service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:13.801335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d33ede-a27a-47c0-9a4f-99a25a02aecd service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:13.801622 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d33ede-a27a-47c0-9a4f-99a25a02aecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:13.803907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-39d33ede-a27a-47c0-9a4f-99a25a02aecd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:13.818604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:13.869810 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d33ede-a27a-47c0-9a4f-99a25a02aecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:13.871500 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d33ede-a27a-47c0-9a4f-99a25a02aecd service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:13.881049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1305/2673] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:14.060974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3cd0f-3077-433b-b232-41c6b573b078 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:14.062541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2bd3cd0f-3077-433b-b232-41c6b573b078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:14.078606 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3cd0f-3077-433b-b232-41c6b573b078 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:14.084145 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:14.085061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:14.085349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:14.085630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] PATH_INFO: `/v3/projects/2d1d02e4165a4631b89f87a19cb7ae74` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:14.086835 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d1d02e4165a4631b89f87a19cb7ae74)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:14.088952 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:14.136789 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d1d02e4165a4631b89f87a19cb7ae74', 'request_id': 'req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.143996 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d1d02e4165a4631b89f87a19cb7ae74', 'request_id': 'req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.149935 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d1d02e4165a4631b89f87a19cb7ae74', 'request_id': 'req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.164133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] The token cache is being invalidate because project 2d1d02e4165a4631b89f87a19cb7ae74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:14.164619 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.165325 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.166904 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1369/2674] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:14 2022] DELETE /identity/v3/projects/2d1d02e4165a4631b89f87a19cb7ae74 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:14.171365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d33ede-a27a-47c0-9a4f-99a25a02aecd service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:14.183928 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d33ede-a27a-47c0-9a4f-99a25a02aecd service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:14.238450 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:14.239323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:14.239603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:14.239875 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] PATH_INFO: `/v3/users/de76c3644e0a4b52a4afd78d67e3c66a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:14.241119 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de76c3644e0a4b52a4afd78d67e3c66a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:14.243032 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:14.288922 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de76c3644e0a4b52a4afd78d67e3c66a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.289575 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de76c3644e0a4b52a4afd78d67e3c66a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.290182 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de76c3644e0a4b52a4afd78d67e3c66a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.302566 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de76c3644e0a4b52a4afd78d67e3c66a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.309819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de76c3644e0a4b52a4afd78d67e3c66a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.327153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1306/2675] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:14 2022] DELETE /identity/v3/users/de76c3644e0a4b52a4afd78d67e3c66a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:14.614765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:14.616229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:14.630515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cb84e8-0dd1-4423-9805-bd2a02a3fc0c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:14.688351 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc922f-6b13-4982-9b91-e38ed46acc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:14.689236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc922f-6b13-4982-9b91-e38ed46acc94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:14.689522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc922f-6b13-4982-9b91-e38ed46acc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:14.689799 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc922f-6b13-4982-9b91-e38ed46acc94 admin admin] PATH_INFO: `/v3/projects/6c458589408342b3867e6520b6a1e291` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:14.690975 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc922f-6b13-4982-9b91-e38ed46acc94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c458589408342b3867e6520b6a1e291)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:14.692894 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc922f-6b13-4982-9b91-e38ed46acc94 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:14.737599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aefc922f-6b13-4982-9b91-e38ed46acc94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c458589408342b3867e6520b6a1e291', 'request_id': 'req-aefc922f-6b13-4982-9b91-e38ed46acc94'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.744931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aefc922f-6b13-4982-9b91-e38ed46acc94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c458589408342b3867e6520b6a1e291', 'request_id': 'req-aefc922f-6b13-4982-9b91-e38ed46acc94'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.751216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aefc922f-6b13-4982-9b91-e38ed46acc94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c458589408342b3867e6520b6a1e291', 'request_id': 'req-aefc922f-6b13-4982-9b91-e38ed46acc94'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.762169 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aefc922f-6b13-4982-9b91-e38ed46acc94 admin admin] The token cache is being invalidate because project 6c458589408342b3867e6520b6a1e291 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:14.762454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aefc922f-6b13-4982-9b91-e38ed46acc94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.763096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-aefc922f-6b13-4982-9b91-e38ed46acc94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:14.764639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1370/2676] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:14 2022] DELETE /identity/v3/projects/6c458589408342b3867e6520b6a1e291 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:23.048818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:23.050371 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:23.064219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994648cb-d4df-4dde-af34-ff4d1f50bde6 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:23.103649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:23.131236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:23.134272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:23.134272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:23.134272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] PATH_INFO: `/v3/users/46a28397a8ea4f0bbfdbbb6e1057e29c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:23.134272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46a28397a8ea4f0bbfdbbb6e1057e29c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:23.138079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:23.189642 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46a28397a8ea4f0bbfdbbb6e1057e29c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.191175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46a28397a8ea4f0bbfdbbb6e1057e29c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.191929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46a28397a8ea4f0bbfdbbb6e1057e29c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.206200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46a28397a8ea4f0bbfdbbb6e1057e29c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.213753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46a28397a8ea4f0bbfdbbb6e1057e29c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.229524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1307/2677] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:23 2022] DELETE /identity/v3/users/46a28397a8ea4f0bbfdbbb6e1057e29c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:23.234776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d948b-08ff-49b3-a43c-262a2d9917ea None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:23.235249 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d948b-08ff-49b3-a43c-262a2d9917ea None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:23.235359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d948b-08ff-49b3-a43c-262a2d9917ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:23.243190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:23.257125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-bd0d948b-08ff-49b3-a43c-262a2d9917ea None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:23.258894 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bd0d948b-08ff-49b3-a43c-262a2d9917ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:23.317510 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1371/2678] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:23.331934 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:23.333805 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:23.397390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229cdba-9701-4df3-b5a3-0491411a22e4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:23.401492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ca8ae-4bda-46f6-b27a-bd4c362451c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:23.402668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ca8ae-4bda-46f6-b27a-bd4c362451c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:23.402949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ca8ae-4bda-46f6-b27a-bd4c362451c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:23.403245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ca8ae-4bda-46f6-b27a-bd4c362451c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:23.414722 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ca8ae-4bda-46f6-b27a-bd4c362451c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:23.416285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ca8ae-4bda-46f6-b27a-bd4c362451c7 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:23.428358 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1308/2679] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:23.706881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d948b-08ff-49b3-a43c-262a2d9917ea None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:23.708603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bd0d948b-08ff-49b3-a43c-262a2d9917ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:23.734069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d948b-08ff-49b3-a43c-262a2d9917ea None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:23.739207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:23.740071 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:23.740359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:23.740641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] PATH_INFO: `/v3/projects/6b9ac089fd32463b8fca23536dbf2944` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:23.741876 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b9ac089fd32463b8fca23536dbf2944)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:23.743884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:23.790005 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b9ac089fd32463b8fca23536dbf2944', 'request_id': 'req-913f862c-f7bc-4930-af42-67b2893dd3f6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.798062 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b9ac089fd32463b8fca23536dbf2944', 'request_id': 'req-913f862c-f7bc-4930-af42-67b2893dd3f6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.804080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b9ac089fd32463b8fca23536dbf2944', 'request_id': 'req-913f862c-f7bc-4930-af42-67b2893dd3f6'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.815265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] The token cache is being invalidate because project 6b9ac089fd32463b8fca23536dbf2944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:23.815562 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.816203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.817775 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1372/2680] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:23 2022] DELETE /identity/v3/projects/6b9ac089fd32463b8fca23536dbf2944 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:23.821771 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ca8ae-4bda-46f6-b27a-bd4c362451c7 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:23.832205 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ca8ae-4bda-46f6-b27a-bd4c362451c7 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:23.882069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:23.889217 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:23.890133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:23.890734 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:23.891064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] PATH_INFO: `/v3/users/f6596e1dabe7428bb41b123f10106a5c` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:23.892321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6596e1dabe7428bb41b123f10106a5c)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:23.894682 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:23.950612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6596e1dabe7428bb41b123f10106a5c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.951166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6596e1dabe7428bb41b123f10106a5c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.951761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6596e1dabe7428bb41b123f10106a5c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.965015 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6596e1dabe7428bb41b123f10106a5c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.973704 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6596e1dabe7428bb41b123f10106a5c'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:23.987355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1309/2681] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:23 2022] DELETE /identity/v3/users/f6596e1dabe7428bb41b123f10106a5c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:24.258665 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:24.260232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:24.274736 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f862c-f7bc-4930-af42-67b2893dd3f6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:24.347834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302378bd-ec3c-4705-aa69-a39f4105f07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:24.348751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302378bd-ec3c-4705-aa69-a39f4105f07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:24.349042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302378bd-ec3c-4705-aa69-a39f4105f07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:24.349323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302378bd-ec3c-4705-aa69-a39f4105f07f admin admin] PATH_INFO: `/v3/projects/0bd73e82b5524fad995417c033d112cc` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:24.350518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302378bd-ec3c-4705-aa69-a39f4105f07f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd73e82b5524fad995417c033d112cc)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:24.354662 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302378bd-ec3c-4705-aa69-a39f4105f07f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:24.362719 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:24.364284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:24.387089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:24.406030 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-302378bd-ec3c-4705-aa69-a39f4105f07f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd73e82b5524fad995417c033d112cc', 'request_id': 'req-302378bd-ec3c-4705-aa69-a39f4105f07f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:24.413621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-302378bd-ec3c-4705-aa69-a39f4105f07f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd73e82b5524fad995417c033d112cc', 'request_id': 'req-302378bd-ec3c-4705-aa69-a39f4105f07f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:24.419413 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-302378bd-ec3c-4705-aa69-a39f4105f07f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd73e82b5524fad995417c033d112cc', 'request_id': 'req-302378bd-ec3c-4705-aa69-a39f4105f07f'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:24.427634 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511a43c-19e9-4bce-b812-4352a1755cc2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:24.431429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-302378bd-ec3c-4705-aa69-a39f4105f07f admin admin] The token cache is being invalidate because project 0bd73e82b5524fad995417c033d112cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:24.431754 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-302378bd-ec3c-4705-aa69-a39f4105f07f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:24.432916 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-302378bd-ec3c-4705-aa69-a39f4105f07f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:24.434784 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1373/2682] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:24 2022] DELETE /identity/v3/projects/0bd73e82b5524fad995417c033d112cc => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:24.439300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91eef3-7fc0-442f-bc3c-665c21daf0c1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:24.439589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91eef3-7fc0-442f-bc3c-665c21daf0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:24.439878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91eef3-7fc0-442f-bc3c-665c21daf0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:24.458667 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-fd91eef3-7fc0-442f-bc3c-665c21daf0c1 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:24.459805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-fd91eef3-7fc0-442f-bc3c-665c21daf0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:24.479638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe9300-b9d1-42a9-ae1b-2a7f490d5947 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:24.480315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe9300-b9d1-42a9-ae1b-2a7f490d5947 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:24.480598 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe9300-b9d1-42a9-ae1b-2a7f490d5947 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:24.480902 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe9300-b9d1-42a9-ae1b-2a7f490d5947 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:24.482700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b0fe9300-b9d1-42a9-ae1b-2a7f490d5947 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:24.525050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1374/2683] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:24.532347 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:24.533392 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcdbec-f73a-47bc-835f-a313379e5f36 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:24.533693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcdbec-f73a-47bc-835f-a313379e5f36 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:24.533998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bcdbec-f73a-47bc-835f-a313379e5f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:24.552577 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe9300-b9d1-42a9-ae1b-2a7f490d5947 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:24.553680 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c2bcdbec-f73a-47bc-835f-a313379e5f36 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:24.554081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fe9300-b9d1-42a9-ae1b-2a7f490d5947 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:24.554906 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c2bcdbec-f73a-47bc-835f-a313379e5f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:24.562968 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1375/2684] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:24.564259 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1310/2685] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:03:24.567095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcdbec-f73a-47bc-835f-a313379e5f36 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:24.577123 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bcdbec-f73a-47bc-835f-a313379e5f36 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:24.581861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c154a9-b7f5-4fb7-a38f-a8eaa6ce6732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:24.582519 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c154a9-b7f5-4fb7-a38f-a8eaa6ce6732 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:24.582810 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c154a9-b7f5-4fb7-a38f-a8eaa6ce6732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:24.583092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c154a9-b7f5-4fb7-a38f-a8eaa6ce6732 admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:24.583821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c154a9-b7f5-4fb7-a38f-a8eaa6ce6732 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:03:24.584134 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c154a9-b7f5-4fb7-a38f-a8eaa6ce6732 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:24.585859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c154a9-b7f5-4fb7-a38f-a8eaa6ce6732 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:24.598324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1376/2686] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:03:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 12:03:24.602750 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe9300-b9d1-42a9-ae1b-2a7f490d5947 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:24.612777 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe9300-b9d1-42a9-ae1b-2a7f490d5947 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:24.617937 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe03b-f0c7-4051-8333-49e5ed0c72a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:24.618657 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe03b-f0c7-4051-8333-49e5ed0c72a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:24.619014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe03b-f0c7-4051-8333-49e5ed0c72a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:24.619466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe03b-f0c7-4051-8333-49e5ed0c72a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:24.620251 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afe03b-f0c7-4051-8333-49e5ed0c72a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:24.622295 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afe03b-f0c7-4051-8333-49e5ed0c72a5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:24.652902 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1311/2687] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:24 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:03:24.654056 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c154a9-b7f5-4fb7-a38f-a8eaa6ce6732 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:24.665201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c154a9-b7f5-4fb7-a38f-a8eaa6ce6732 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:24.724559 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683abc5-94be-432d-b473-36770f6ed207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:24.725262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683abc5-94be-432d-b473-36770f6ed207 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:24.725581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683abc5-94be-432d-b473-36770f6ed207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:24.725838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683abc5-94be-432d-b473-36770f6ed207 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:24.726527 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683abc5-94be-432d-b473-36770f6ed207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:24.728435 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683abc5-94be-432d-b473-36770f6ed207 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:24.751415 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1377/2688] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:24 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:03:24.755552 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe03b-f0c7-4051-8333-49e5ed0c72a5 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:24.766894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe03b-f0c7-4051-8333-49e5ed0c72a5 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:24.771649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0e18b-99b0-44e5-a79d-85a71f773192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:24.772321 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0e18b-99b0-44e5-a79d-85a71f773192 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:24.772608 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0e18b-99b0-44e5-a79d-85a71f773192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:24.772938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0e18b-99b0-44e5-a79d-85a71f773192 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:24.773671 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0e18b-99b0-44e5-a79d-85a71f773192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:24.775276 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0e18b-99b0-44e5-a79d-85a71f773192 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:24.786336 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1312/2689] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:24.790236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683abc5-94be-432d-b473-36770f6ed207 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:24.801836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683abc5-94be-432d-b473-36770f6ed207 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:24.806702 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c943bc9-429f-44b6-9692-f1c6fcdff0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:24.807372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c943bc9-429f-44b6-9692-f1c6fcdff0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:24.807660 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c943bc9-429f-44b6-9692-f1c6fcdff0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:24.807940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c943bc9-429f-44b6-9692-f1c6fcdff0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:24.809269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c943bc9-429f-44b6-9692-f1c6fcdff0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:24.810866 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c943bc9-429f-44b6-9692-f1c6fcdff0c3 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:24.821205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1378/2690] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:24.825921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0e18b-99b0-44e5-a79d-85a71f773192 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:24.836092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0e18b-99b0-44e5-a79d-85a71f773192 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:24.841623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aeb6a6-720e-4951-b94f-55c719f7c351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:24.842505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aeb6a6-720e-4951-b94f-55c719f7c351 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:24.842793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aeb6a6-720e-4951-b94f-55c719f7c351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:24.843073 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aeb6a6-720e-4951-b94f-55c719f7c351 admin admin] PATH_INFO: `/v3/projects/953322cb89724d80b6773b68c4575ba9/users/2fcba93034634079b69c64dcbdc4e81a/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:24.859506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aeb6a6-720e-4951-b94f-55c719f7c351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=953322cb89724d80b6773b68c4575ba9, user_id=2fcba93034634079b69c64dcbdc4e81a, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:24.862226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aeb6a6-720e-4951-b94f-55c719f7c351 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:24.875865 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1313/2691] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:24 2022] PUT /identity/v3/projects/953322cb89724d80b6773b68c4575ba9/users/2fcba93034634079b69c64dcbdc4e81a/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:24.886205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1795b-d6ae-4a8c-b8fe-46a7f833dbe1 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:24.886533 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1795b-d6ae-4a8c-b8fe-46a7f833dbe1 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:24.886822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1795b-d6ae-4a8c-b8fe-46a7f833dbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:24.914029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-40d1795b-d6ae-4a8c-b8fe-46a7f833dbe1 None None] MFA Rules not processed for user `2fcba93034634079b69c64dcbdc4e81a`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:24.915621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-40d1795b-d6ae-4a8c-b8fe-46a7f833dbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:24.920307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:24.990602 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1379/2692] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:24.994759 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aeb6a6-720e-4951-b94f-55c719f7c351 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:25.057562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aeb6a6-720e-4951-b94f-55c719f7c351 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:25.062288 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e957-854b-4513-b42f-4a62a5230603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:25.062942 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e957-854b-4513-b42f-4a62a5230603 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:25.063243 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e957-854b-4513-b42f-4a62a5230603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:25.063501 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389e957-854b-4513-b42f-4a62a5230603 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:25.064199 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389e957-854b-4513-b42f-4a62a5230603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:25.066164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389e957-854b-4513-b42f-4a62a5230603 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:25.101982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1314/2693] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:24 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:03:25.103177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1795b-d6ae-4a8c-b8fe-46a7f833dbe1 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:25.115161 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1795b-d6ae-4a8c-b8fe-46a7f833dbe1 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:25.172812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924e910-7d4a-4b0e-8c26-67675337922e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:25.173495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924e910-7d4a-4b0e-8c26-67675337922e admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:25.173775 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924e910-7d4a-4b0e-8c26-67675337922e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:25.174054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2924e910-7d4a-4b0e-8c26-67675337922e admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:25.174742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924e910-7d4a-4b0e-8c26-67675337922e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:25.176977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2924e910-7d4a-4b0e-8c26-67675337922e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:25.198192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1380/2694] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:25 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:25.202305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e957-854b-4513-b42f-4a62a5230603 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:25.213473 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389e957-854b-4513-b42f-4a62a5230603 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:25.218262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d99321-a7ae-49c7-9cb9-bd9fcf77d30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:25.218926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d99321-a7ae-49c7-9cb9-bd9fcf77d30f admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:25.219215 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d99321-a7ae-49c7-9cb9-bd9fcf77d30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:25.219495 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d99321-a7ae-49c7-9cb9-bd9fcf77d30f admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:25.220227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d99321-a7ae-49c7-9cb9-bd9fcf77d30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:25.221849 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d99321-a7ae-49c7-9cb9-bd9fcf77d30f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:25.233761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1315/2695] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:25.241568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924e910-7d4a-4b0e-8c26-67675337922e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:25.252896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2924e910-7d4a-4b0e-8c26-67675337922e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:25.257711 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb90b80-b9cc-4d39-869a-3efc6bb387ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:25.258630 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb90b80-b9cc-4d39-869a-3efc6bb387ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:25.258921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb90b80-b9cc-4d39-869a-3efc6bb387ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:25.259472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb90b80-b9cc-4d39-869a-3efc6bb387ee admin admin] PATH_INFO: `/v3/projects/54321bf12081492dafebdb8b7bce8d18/users/eb5391ef526b4154ac8b1a5fb83ee079/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:25.274374 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb90b80-b9cc-4d39-869a-3efc6bb387ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=54321bf12081492dafebdb8b7bce8d18, user_id=eb5391ef526b4154ac8b1a5fb83ee079, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:25.277080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb90b80-b9cc-4d39-869a-3efc6bb387ee admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:25.290106 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1381/2696] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:25 2022] PUT /identity/v3/projects/54321bf12081492dafebdb8b7bce8d18/users/eb5391ef526b4154ac8b1a5fb83ee079/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:25.294536 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d99321-a7ae-49c7-9cb9-bd9fcf77d30f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:25.305007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d99321-a7ae-49c7-9cb9-bd9fcf77d30f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:25.367197 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91a66d-5f29-48e6-9a3a-35746faa1e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:25.368067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91a66d-5f29-48e6-9a3a-35746faa1e07 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:25.368328 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91a66d-5f29-48e6-9a3a-35746faa1e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:25.368611 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91a66d-5f29-48e6-9a3a-35746faa1e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:25.369393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91a66d-5f29-48e6-9a3a-35746faa1e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:25.371595 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91a66d-5f29-48e6-9a3a-35746faa1e07 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:25.373053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:25.384710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1316/2697] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:25 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:25.388728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb90b80-b9cc-4d39-869a-3efc6bb387ee admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:25.399319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb90b80-b9cc-4d39-869a-3efc6bb387ee admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:25.404045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc663b-a7d3-4a59-8011-ced92d2b7069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:25.404963 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc663b-a7d3-4a59-8011-ced92d2b7069 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:25.405245 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc663b-a7d3-4a59-8011-ced92d2b7069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:25.405521 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fc663b-a7d3-4a59-8011-ced92d2b7069 admin admin] PATH_INFO: `/v3/projects/54321bf12081492dafebdb8b7bce8d18/users/eb5391ef526b4154ac8b1a5fb83ee079/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:25.407560 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc663b-a7d3-4a59-8011-ced92d2b7069 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54321bf12081492dafebdb8b7bce8d18, user_id=eb5391ef526b4154ac8b1a5fb83ee079, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:25.410239 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fc663b-a7d3-4a59-8011-ced92d2b7069 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:25.421811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1382/2698] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:25 2022] PUT /identity/v3/projects/54321bf12081492dafebdb8b7bce8d18/users/eb5391ef526b4154ac8b1a5fb83ee079/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:25.434152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d58ac-aa12-416e-8a53-caf2518b6970 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:25.434444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d58ac-aa12-416e-8a53-caf2518b6970 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:25.434729 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d58ac-aa12-416e-8a53-caf2518b6970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:25.467596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-584d58ac-aa12-416e-8a53-caf2518b6970 None None] MFA Rules not processed for user `eb5391ef526b4154ac8b1a5fb83ee079`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:25.468896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-584d58ac-aa12-416e-8a53-caf2518b6970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:25.550001 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1317/2699] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:25.558345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc663b-a7d3-4a59-8011-ced92d2b7069 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:25.560416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-14fc663b-a7d3-4a59-8011-ced92d2b7069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:25.642147 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fc663b-a7d3-4a59-8011-ced92d2b7069 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:25.645947 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d1a60-ccae-4e63-be99-370ff9ebc5f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:25.646691 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d1a60-ccae-4e63-be99-370ff9ebc5f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:25.646988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d1a60-ccae-4e63-be99-370ff9ebc5f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:25.647299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d1a60-ccae-4e63-be99-370ff9ebc5f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:25.661515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d1a60-ccae-4e63-be99-370ff9ebc5f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:25.665247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d1a60-ccae-4e63-be99-370ff9ebc5f7 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:25.671609 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:25.677403 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1383/2700] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:25.744160 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d58ac-aa12-416e-8a53-caf2518b6970 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:25.816492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d58ac-aa12-416e-8a53-caf2518b6970 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:25.821214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:25.822102 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:25.822533 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:25.822818 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] PATH_INFO: `/v3/users/2fcba93034634079b69c64dcbdc4e81a` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:25.824609 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fcba93034634079b69c64dcbdc4e81a)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:25.826588 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:25.880827 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fcba93034634079b69c64dcbdc4e81a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:25.881507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fcba93034634079b69c64dcbdc4e81a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:25.882117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fcba93034634079b69c64dcbdc4e81a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:25.895272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fcba93034634079b69c64dcbdc4e81a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:25.902903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fcba93034634079b69c64dcbdc4e81a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:25.913343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:25.920761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1318/2701] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:25 2022] DELETE /identity/v3/users/2fcba93034634079b69c64dcbdc4e81a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:25.923159 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d1526-d301-4650-b85f-c59af06c4d39 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:25.923478 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d1526-d301-4650-b85f-c59af06c4d39 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:25.923762 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d1526-d301-4650-b85f-c59af06c4d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:25.942818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-a12d1526-d301-4650-b85f-c59af06c4d39 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:25.944063 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a12d1526-d301-4650-b85f-c59af06c4d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:26.016744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1384/2702] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:26.023731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:26.025218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:26.084395 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74500934-d190-4a16-b070-c2e04410ef33 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:26.088099 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14ea51-9409-4860-9bcc-c923304cbd74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:26.088781 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14ea51-9409-4860-9bcc-c923304cbd74 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:26.089067 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14ea51-9409-4860-9bcc-c923304cbd74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:26.089901 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14ea51-9409-4860-9bcc-c923304cbd74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:26.099238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14ea51-9409-4860-9bcc-c923304cbd74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:26.100728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14ea51-9409-4860-9bcc-c923304cbd74 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:26.110163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1319/2703] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:26.410232 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d1526-d301-4650-b85f-c59af06c4d39 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:26.411992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-a12d1526-d301-4650-b85f-c59af06c4d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:26.426872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d1526-d301-4650-b85f-c59af06c4d39 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:26.432458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:26.434052 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:26.435016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:26.435406 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] PATH_INFO: `/v3/projects/953322cb89724d80b6773b68c4575ba9` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:26.436796 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=953322cb89724d80b6773b68c4575ba9)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:26.439310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:26.527753 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '953322cb89724d80b6773b68c4575ba9', 'request_id': 'req-00d99d38-9edf-410b-8c57-617ece1a1e34'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:26.540259 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '953322cb89724d80b6773b68c4575ba9', 'request_id': 'req-00d99d38-9edf-410b-8c57-617ece1a1e34'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:26.556640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '953322cb89724d80b6773b68c4575ba9', 'request_id': 'req-00d99d38-9edf-410b-8c57-617ece1a1e34'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:26.573144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] The token cache is being invalidate because project 953322cb89724d80b6773b68c4575ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:26.573515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:26.574161 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:26.576000 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1385/2704] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:26 2022] DELETE /identity/v3/projects/953322cb89724d80b6773b68c4575ba9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:26.580285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14ea51-9409-4860-9bcc-c923304cbd74 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:26.594588 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14ea51-9409-4860-9bcc-c923304cbd74 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:26.675267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010403a-6b6b-495f-bb71-0271057f587f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:26.676593 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010403a-6b6b-495f-bb71-0271057f587f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:26.676896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010403a-6b6b-495f-bb71-0271057f587f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:26.677181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010403a-6b6b-495f-bb71-0271057f587f admin admin] PATH_INFO: `/v3/users/eb5391ef526b4154ac8b1a5fb83ee079` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:26.678455 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010403a-6b6b-495f-bb71-0271057f587f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb5391ef526b4154ac8b1a5fb83ee079)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:26.680404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b010403a-6b6b-495f-bb71-0271057f587f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:26.738770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b010403a-6b6b-495f-bb71-0271057f587f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb5391ef526b4154ac8b1a5fb83ee079'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:26.740199 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b010403a-6b6b-495f-bb71-0271057f587f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb5391ef526b4154ac8b1a5fb83ee079'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:26.741070 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b010403a-6b6b-495f-bb71-0271057f587f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb5391ef526b4154ac8b1a5fb83ee079'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:26.757860 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b010403a-6b6b-495f-bb71-0271057f587f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb5391ef526b4154ac8b1a5fb83ee079'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:26.767905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b010403a-6b6b-495f-bb71-0271057f587f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb5391ef526b4154ac8b1a5fb83ee079'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:26.793278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1320/2705] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:26 2022] DELETE /identity/v3/users/eb5391ef526b4154ac8b1a5fb83ee079 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:27.119604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:27.121832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:27.141206 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d99d38-9edf-410b-8c57-617ece1a1e34 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:27.215563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dac5a7-9989-487e-95a9-34c6524020fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:27.216805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dac5a7-9989-487e-95a9-34c6524020fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:27.217191 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dac5a7-9989-487e-95a9-34c6524020fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:27.217517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dac5a7-9989-487e-95a9-34c6524020fa admin admin] PATH_INFO: `/v3/projects/54321bf12081492dafebdb8b7bce8d18` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:27.218822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dac5a7-9989-487e-95a9-34c6524020fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=54321bf12081492dafebdb8b7bce8d18)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:27.220891 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dac5a7-9989-487e-95a9-34c6524020fa admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:27.264505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:27.287580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-74dac5a7-9989-487e-95a9-34c6524020fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54321bf12081492dafebdb8b7bce8d18', 'request_id': 'req-74dac5a7-9989-487e-95a9-34c6524020fa'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:27.302243 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-74dac5a7-9989-487e-95a9-34c6524020fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54321bf12081492dafebdb8b7bce8d18', 'request_id': 'req-74dac5a7-9989-487e-95a9-34c6524020fa'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:27.318338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-74dac5a7-9989-487e-95a9-34c6524020fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54321bf12081492dafebdb8b7bce8d18', 'request_id': 'req-74dac5a7-9989-487e-95a9-34c6524020fa'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:27.331087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-74dac5a7-9989-487e-95a9-34c6524020fa admin admin] The token cache is being invalidate because project 54321bf12081492dafebdb8b7bce8d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:27.331485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-74dac5a7-9989-487e-95a9-34c6524020fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:27.332228 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-74dac5a7-9989-487e-95a9-34c6524020fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:27.334191 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1386/2706] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:27 2022] DELETE /identity/v3/projects/54321bf12081492dafebdb8b7bce8d18 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:27.338811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf6302-be5c-47df-bf01-5a5c216eb667 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:27.339100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf6302-be5c-47df-bf01-5a5c216eb667 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:27.339448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf6302-be5c-47df-bf01-5a5c216eb667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:27.358474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9edf6302-be5c-47df-bf01-5a5c216eb667 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:27.359821 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9edf6302-be5c-47df-bf01-5a5c216eb667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:27.377418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:27.435357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1321/2707] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:27.443787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f4871-9507-410e-91de-e71986775371 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:27.444321 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f4871-9507-410e-91de-e71986775371 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:27.444525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f4871-9507-410e-91de-e71986775371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:27.463730 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-639f4871-9507-410e-91de-e71986775371 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:27.465553 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-639f4871-9507-410e-91de-e71986775371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:27.473636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1387/2708] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:27.477850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf6302-be5c-47df-bf01-5a5c216eb667 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:27.488107 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf6302-be5c-47df-bf01-5a5c216eb667 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:27.492918 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbca5c8-285e-43ec-b67d-33255f52f9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:27.493580 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbca5c8-285e-43ec-b67d-33255f52f9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:27.493863 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbca5c8-285e-43ec-b67d-33255f52f9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:27.494149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbca5c8-285e-43ec-b67d-33255f52f9cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:27.494873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbca5c8-285e-43ec-b67d-33255f52f9cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:03:27.495181 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbca5c8-285e-43ec-b67d-33255f52f9cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:27.496840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbca5c8-285e-43ec-b67d-33255f52f9cc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:27.513632 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1322/2709] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:03:27.515013 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f4871-9507-410e-91de-e71986775371 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:27.525171 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f4871-9507-410e-91de-e71986775371 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:27.530761 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7192b-8370-4746-9d62-e23262dcdbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:27.531457 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7192b-8370-4746-9d62-e23262dcdbaa admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:27.531740 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7192b-8370-4746-9d62-e23262dcdbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:27.532025 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7192b-8370-4746-9d62-e23262dcdbaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:27.532748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7192b-8370-4746-9d62-e23262dcdbaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:27.534647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7192b-8370-4746-9d62-e23262dcdbaa admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:27.565643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1388/2710] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:27 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:27.566677 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbca5c8-285e-43ec-b67d-33255f52f9cc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:27.577903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbca5c8-285e-43ec-b67d-33255f52f9cc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:27.661218 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bebc63-372e-417d-b53f-1a190e28fada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:27.661900 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bebc63-372e-417d-b53f-1a190e28fada admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:27.662190 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bebc63-372e-417d-b53f-1a190e28fada admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:27.662468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bebc63-372e-417d-b53f-1a190e28fada admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:27.663184 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bebc63-372e-417d-b53f-1a190e28fada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:27.665145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bebc63-372e-417d-b53f-1a190e28fada admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:27.691762 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1323/2711] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:27 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:27.694591 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7192b-8370-4746-9d62-e23262dcdbaa admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:27.706051 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7192b-8370-4746-9d62-e23262dcdbaa admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:27.711978 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18f1b2-b56f-43f3-a7ff-d8844bb9c4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:27.712718 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18f1b2-b56f-43f3-a7ff-d8844bb9c4cf admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:27.713010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18f1b2-b56f-43f3-a7ff-d8844bb9c4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:27.713418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df18f1b2-b56f-43f3-a7ff-d8844bb9c4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:27.714160 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18f1b2-b56f-43f3-a7ff-d8844bb9c4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:27.715878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df18f1b2-b56f-43f3-a7ff-d8844bb9c4cf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:27.729599 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1389/2712] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:27.730792 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bebc63-372e-417d-b53f-1a190e28fada admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:27.742731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bebc63-372e-417d-b53f-1a190e28fada admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:27.748574 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c229433-1f12-4286-b5a4-bdd317acddf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:27.749249 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c229433-1f12-4286-b5a4-bdd317acddf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:27.749542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c229433-1f12-4286-b5a4-bdd317acddf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:27.749822 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c229433-1f12-4286-b5a4-bdd317acddf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:27.750698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c229433-1f12-4286-b5a4-bdd317acddf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:27.752286 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c229433-1f12-4286-b5a4-bdd317acddf2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:27.764316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1324/2713] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:03:27.768551 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18f1b2-b56f-43f3-a7ff-d8844bb9c4cf admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:27.779575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df18f1b2-b56f-43f3-a7ff-d8844bb9c4cf admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:27.784820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6ca9f-c22b-4de4-a124-59892d62107d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:27.785746 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6ca9f-c22b-4de4-a124-59892d62107d admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:27.786028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6ca9f-c22b-4de4-a124-59892d62107d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:27.786307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6ca9f-c22b-4de4-a124-59892d62107d admin admin] PATH_INFO: `/v3/projects/75e59db7e77d41bf9c46aa7c010b41b0/users/07fbea52894e4707bbb1d4f15b42a80e/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:27.802937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6ca9f-c22b-4de4-a124-59892d62107d admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e59db7e77d41bf9c46aa7c010b41b0, user_id=07fbea52894e4707bbb1d4f15b42a80e, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:27.805830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6ca9f-c22b-4de4-a124-59892d62107d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:27.822681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1390/2714] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:27 2022] PUT /identity/v3/projects/75e59db7e77d41bf9c46aa7c010b41b0/users/07fbea52894e4707bbb1d4f15b42a80e/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:27.841538 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6ca9f-c22b-4de4-a124-59892d62107d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:27.842297 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29040172-ca3d-480e-9070-9f3bd4e30ca3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:27.842929 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29040172-ca3d-480e-9070-9f3bd4e30ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:27.843379 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29040172-ca3d-480e-9070-9f3bd4e30ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:27.844363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-bfb6ca9f-c22b-4de4-a124-59892d62107d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:27.891104 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-29040172-ca3d-480e-9070-9f3bd4e30ca3 None None] MFA Rules not processed for user `07fbea52894e4707bbb1d4f15b42a80e`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:27.893086 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-29040172-ca3d-480e-9070-9f3bd4e30ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:27.947059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6ca9f-c22b-4de4-a124-59892d62107d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:27.954547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c9671-d94b-4ffd-9bb8-5410fd1d1123 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:27.955220 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c9671-d94b-4ffd-9bb8-5410fd1d1123 service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:27.955513 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c9671-d94b-4ffd-9bb8-5410fd1d1123 service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:27.955823 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c9671-d94b-4ffd-9bb8-5410fd1d1123 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:27.957856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c13c9671-d94b-4ffd-9bb8-5410fd1d1123 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:27.970958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c9671-d94b-4ffd-9bb8-5410fd1d1123 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:27.972568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c9671-d94b-4ffd-9bb8-5410fd1d1123 service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:27.979166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1325/2715] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:27.980493 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29040172-ca3d-480e-9070-9f3bd4e30ca3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:27.981930 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-29040172-ca3d-480e-9070-9f3bd4e30ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:27.985492 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1391/2716] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:03:27.999912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c9671-d94b-4ffd-9bb8-5410fd1d1123 service placement] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:28.001556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c13c9671-d94b-4ffd-9bb8-5410fd1d1123 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:28.013736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:28.052431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:28.060228 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29040172-ca3d-480e-9070-9f3bd4e30ca3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:28.065301 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d79e45-2e50-42ce-9617-ea0af0473e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:28.066178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d79e45-2e50-42ce-9617-ea0af0473e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:28.066474 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d79e45-2e50-42ce-9617-ea0af0473e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:28.066739 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d79e45-2e50-42ce-9617-ea0af0473e70 admin admin] PATH_INFO: `/v3/users/ebfb5a6c657a4fd592ff9e67348627eb` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:28.068051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d79e45-2e50-42ce-9617-ea0af0473e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebfb5a6c657a4fd592ff9e67348627eb)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:28.068504 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c9671-d94b-4ffd-9bb8-5410fd1d1123 service placement] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:28.069964 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d79e45-2e50-42ce-9617-ea0af0473e70 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:28.072281 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875f4e2-7d6e-465a-bd9d-470983273f40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:28.072959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875f4e2-7d6e-465a-bd9d-470983273f40 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:28.073323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875f4e2-7d6e-465a-bd9d-470983273f40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:28.073517 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875f4e2-7d6e-465a-bd9d-470983273f40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:28.083174 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875f4e2-7d6e-465a-bd9d-470983273f40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:28.084645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875f4e2-7d6e-465a-bd9d-470983273f40 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:28.093962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1392/2717] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:03:28.116738 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-70d79e45-2e50-42ce-9617-ea0af0473e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebfb5a6c657a4fd592ff9e67348627eb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:28.117419 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-70d79e45-2e50-42ce-9617-ea0af0473e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebfb5a6c657a4fd592ff9e67348627eb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:28.118170 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-70d79e45-2e50-42ce-9617-ea0af0473e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebfb5a6c657a4fd592ff9e67348627eb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:28.131016 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-70d79e45-2e50-42ce-9617-ea0af0473e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebfb5a6c657a4fd592ff9e67348627eb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:28.138005 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-70d79e45-2e50-42ce-9617-ea0af0473e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebfb5a6c657a4fd592ff9e67348627eb'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:28.151175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1326/2718] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:27 2022] DELETE /identity/v3/users/ebfb5a6c657a4fd592ff9e67348627eb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:28.345149 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875f4e2-7d6e-465a-bd9d-470983273f40 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:28.371598 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875f4e2-7d6e-465a-bd9d-470983273f40 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:28.452087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d5c2f-bf89-4866-ad74-e219403e6c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:28.452986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5c2f-bf89-4866-ad74-e219403e6c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:28.453298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5c2f-bf89-4866-ad74-e219403e6c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:28.453580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d5c2f-bf89-4866-ad74-e219403e6c73 admin admin] PATH_INFO: `/v3/projects/d211354c655b469491d235e5507e61ba` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:28.454839 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d5c2f-bf89-4866-ad74-e219403e6c73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d211354c655b469491d235e5507e61ba)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:28.456805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d5c2f-bf89-4866-ad74-e219403e6c73 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:28.506241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-614d5c2f-bf89-4866-ad74-e219403e6c73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd211354c655b469491d235e5507e61ba', 'request_id': 'req-614d5c2f-bf89-4866-ad74-e219403e6c73'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:28.515541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-614d5c2f-bf89-4866-ad74-e219403e6c73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd211354c655b469491d235e5507e61ba', 'request_id': 'req-614d5c2f-bf89-4866-ad74-e219403e6c73'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:28.524981 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-614d5c2f-bf89-4866-ad74-e219403e6c73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd211354c655b469491d235e5507e61ba', 'request_id': 'req-614d5c2f-bf89-4866-ad74-e219403e6c73'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:28.536762 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-614d5c2f-bf89-4866-ad74-e219403e6c73 admin admin] The token cache is being invalidate because project d211354c655b469491d235e5507e61ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:28.537059 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-614d5c2f-bf89-4866-ad74-e219403e6c73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:28.537734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-614d5c2f-bf89-4866-ad74-e219403e6c73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:28.539292 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1393/2719] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:28 2022] DELETE /identity/v3/projects/d211354c655b469491d235e5507e61ba => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:28.544426 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19871216-288d-424e-8247-97ca407e868e None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:28.544887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19871216-288d-424e-8247-97ca407e868e None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:28.545214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19871216-288d-424e-8247-97ca407e868e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:28.564733 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-19871216-288d-424e-8247-97ca407e868e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:28.565990 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-19871216-288d-424e-8247-97ca407e868e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:28.628910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1327/2720] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:28.637792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e8845-3f16-4184-9604-61e70e037a85 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:28.638108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e8845-3f16-4184-9604-61e70e037a85 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:28.638473 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e8845-3f16-4184-9604-61e70e037a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:28.658041 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-273e8845-3f16-4184-9604-61e70e037a85 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:28.659364 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-273e8845-3f16-4184-9604-61e70e037a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:28.667892 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1394/2721] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:28.672517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19871216-288d-424e-8247-97ca407e868e None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:28.684126 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19871216-288d-424e-8247-97ca407e868e None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:28.688841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b475929-b5cc-4781-9905-9aa6f4c6617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:28.689523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b475929-b5cc-4781-9905-9aa6f4c6617f admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:28.689820 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b475929-b5cc-4781-9905-9aa6f4c6617f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:28.690105 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b475929-b5cc-4781-9905-9aa6f4c6617f admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:28.690835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b475929-b5cc-4781-9905-9aa6f4c6617f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:03:28.691150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b475929-b5cc-4781-9905-9aa6f4c6617f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:28.693206 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b475929-b5cc-4781-9905-9aa6f4c6617f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:28.706659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1328/2722] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:03:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:03:28.711037 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e8845-3f16-4184-9604-61e70e037a85 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:28.720465 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e8845-3f16-4184-9604-61e70e037a85 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:28.725186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200e09a-449e-460d-9705-40a32a686cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:28.725838 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200e09a-449e-460d-9705-40a32a686cb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:28.726127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200e09a-449e-460d-9705-40a32a686cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:28.726409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d200e09a-449e-460d-9705-40a32a686cb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:28.727100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200e09a-449e-460d-9705-40a32a686cb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:28.729085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d200e09a-449e-460d-9705-40a32a686cb4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:28.755126 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1395/2723] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:28 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:28.759289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b475929-b5cc-4781-9905-9aa6f4c6617f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:28.769886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b475929-b5cc-4781-9905-9aa6f4c6617f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:28.789701 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:28.826582 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068dd08-e49c-4de6-a649-f91aa4be4692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:28.827255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068dd08-e49c-4de6-a649-f91aa4be4692 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:28.827541 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068dd08-e49c-4de6-a649-f91aa4be4692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:28.827823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068dd08-e49c-4de6-a649-f91aa4be4692 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:28.828546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068dd08-e49c-4de6-a649-f91aa4be4692 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:28.830459 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068dd08-e49c-4de6-a649-f91aa4be4692 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:28.849905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1329/2724] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:28 2022] POST /identity/v3/users => generated 286 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:28.854042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200e09a-449e-460d-9705-40a32a686cb4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:28.864914 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d200e09a-449e-460d-9705-40a32a686cb4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:28.869587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd82a5-fcb2-4cfb-8b02-c974b902f07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:28.870241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd82a5-fcb2-4cfb-8b02-c974b902f07e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:28.870645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd82a5-fcb2-4cfb-8b02-c974b902f07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:28.871434 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd82a5-fcb2-4cfb-8b02-c974b902f07e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:28.872163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd82a5-fcb2-4cfb-8b02-c974b902f07e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:28.873797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd82a5-fcb2-4cfb-8b02-c974b902f07e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:28.885195 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1396/2725] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:28.889194 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068dd08-e49c-4de6-a649-f91aa4be4692 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:28.903510 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068dd08-e49c-4de6-a649-f91aa4be4692 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:28.908257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5182a36-eecf-457b-bf9d-680cfa539a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:28.908962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5182a36-eecf-457b-bf9d-680cfa539a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:28.909248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5182a36-eecf-457b-bf9d-680cfa539a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:28.909526 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5182a36-eecf-457b-bf9d-680cfa539a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:28.910267 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5182a36-eecf-457b-bf9d-680cfa539a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:28.911840 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5182a36-eecf-457b-bf9d-680cfa539a51 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:28.923105 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1330/2726] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:28.927669 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd82a5-fcb2-4cfb-8b02-c974b902f07e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:28.938464 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd82a5-fcb2-4cfb-8b02-c974b902f07e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:28.943447 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e77b5-1154-495f-b6d5-fd76d91831e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:28.944323 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e77b5-1154-495f-b6d5-fd76d91831e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:28.944638 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e77b5-1154-495f-b6d5-fd76d91831e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:28.944962 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e77b5-1154-495f-b6d5-fd76d91831e6 admin admin] PATH_INFO: `/v3/projects/0a78150292244570817a5eb92674adff/users/666354fb1f144814b8b5b462005b7c55/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:28.959423 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e77b5-1154-495f-b6d5-fd76d91831e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a78150292244570817a5eb92674adff, user_id=666354fb1f144814b8b5b462005b7c55, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:28.962111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e77b5-1154-495f-b6d5-fd76d91831e6 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:28.975789 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1397/2727] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:28 2022] PUT /identity/v3/projects/0a78150292244570817a5eb92674adff/users/666354fb1f144814b8b5b462005b7c55/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:28.987224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25401e-af13-4235-a93d-54fa8b091b18 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:28.987530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25401e-af13-4235-a93d-54fa8b091b18 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:28.987914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25401e-af13-4235-a93d-54fa8b091b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:29.014978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-cb25401e-af13-4235-a93d-54fa8b091b18 None None] MFA Rules not processed for user `666354fb1f144814b8b5b462005b7c55`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:29.016286 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-cb25401e-af13-4235-a93d-54fa8b091b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:29.080175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1331/2728] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:29.091462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e77b5-1154-495f-b6d5-fd76d91831e6 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:29.093290 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-680e77b5-1154-495f-b6d5-fd76d91831e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:29.104183 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:29.173253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e77b5-1154-495f-b6d5-fd76d91831e6 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:29.179125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687e6bb-270f-42c7-8462-e07cbc81294e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:29.179776 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687e6bb-270f-42c7-8462-e07cbc81294e service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:29.180070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687e6bb-270f-42c7-8462-e07cbc81294e service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:29.180357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687e6bb-270f-42c7-8462-e07cbc81294e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:29.189024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687e6bb-270f-42c7-8462-e07cbc81294e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:29.190633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687e6bb-270f-42c7-8462-e07cbc81294e service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:29.199587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1398/2729] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:29.412919 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25401e-af13-4235-a93d-54fa8b091b18 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:29.477896 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25401e-af13-4235-a93d-54fa8b091b18 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:29.483253 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:29.484206 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:29.484490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:29.484892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] PATH_INFO: `/v3/users/666354fb1f144814b8b5b462005b7c55` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:29.486088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=666354fb1f144814b8b5b462005b7c55)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:29.487991 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:29.529152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:29.540208 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '666354fb1f144814b8b5b462005b7c55'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:29.540887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '666354fb1f144814b8b5b462005b7c55'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:29.541493 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '666354fb1f144814b8b5b462005b7c55'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:29.559109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '666354fb1f144814b8b5b462005b7c55'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:29.566186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '666354fb1f144814b8b5b462005b7c55'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:29.581031 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1332/2730] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:29 2022] DELETE /identity/v3/users/666354fb1f144814b8b5b462005b7c55 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:29.585401 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e2e8d-3cfa-4e2c-bf3d-db3eacdcc0e6 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:29.585894 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e2e8d-3cfa-4e2c-bf3d-db3eacdcc0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:29.585980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e2e8d-3cfa-4e2c-bf3d-db3eacdcc0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:29.605856 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-c16e2e8d-3cfa-4e2c-bf3d-db3eacdcc0e6 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:29.607115 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c16e2e8d-3cfa-4e2c-bf3d-db3eacdcc0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:29.675247 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1399/2731] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:29.685427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:29.686923 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:29.748467 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932b54bc-b96c-486d-8829-a8fcf8ba9c0f admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:29.752476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356c034-a235-4f19-90f9-0068ee0e79ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:29.753174 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356c034-a235-4f19-90f9-0068ee0e79ed service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:29.753441 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356c034-a235-4f19-90f9-0068ee0e79ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:29.753722 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356c034-a235-4f19-90f9-0068ee0e79ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:29.763753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356c034-a235-4f19-90f9-0068ee0e79ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:29.765234 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356c034-a235-4f19-90f9-0068ee0e79ed service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:29.775684 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1333/2732] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:30.066564 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e2e8d-3cfa-4e2c-bf3d-db3eacdcc0e6 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:30.068028 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c16e2e8d-3cfa-4e2c-bf3d-db3eacdcc0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:30.082161 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e2e8d-3cfa-4e2c-bf3d-db3eacdcc0e6 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:30.088179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:30.089387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.089713 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.090095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d admin admin] PATH_INFO: `/v3/projects/0a78150292244570817a5eb92674adff` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.091487 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a78150292244570817a5eb92674adff)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:30.093449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:30.142130 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a78150292244570817a5eb92674adff', 'request_id': 'req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:30.149497 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a78150292244570817a5eb92674adff', 'request_id': 'req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:30.156256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a78150292244570817a5eb92674adff', 'request_id': 'req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:30.167774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d admin admin] The token cache is being invalidate because project 0a78150292244570817a5eb92674adff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:30.168070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:30.168769 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-bdd95cec-9d5a-4460-a32a-ff2bede21b9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:30.170354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1400/2733] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:30 2022] DELETE /identity/v3/projects/0a78150292244570817a5eb92674adff => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:30.175177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7007d40-0684-4bef-8f2f-4921bb587d16 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.175466 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7007d40-0684-4bef-8f2f-4921bb587d16 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.176012 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7007d40-0684-4bef-8f2f-4921bb587d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.193902 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-c7007d40-0684-4bef-8f2f-4921bb587d16 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:30.195125 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c7007d40-0684-4bef-8f2f-4921bb587d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:30.257912 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1334/2734] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:30.266572 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcbbeb-ea55-45c8-9c0f-d73b7b5ffc0f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.266985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcbbeb-ea55-45c8-9c0f-d73b7b5ffc0f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.267446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcbbeb-ea55-45c8-9c0f-d73b7b5ffc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.289040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b2dcbbeb-ea55-45c8-9c0f-d73b7b5ffc0f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:30.290275 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b2dcbbeb-ea55-45c8-9c0f-d73b7b5ffc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:30.299909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1401/2735] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:30.304316 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7007d40-0684-4bef-8f2f-4921bb587d16 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:30.314648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7007d40-0684-4bef-8f2f-4921bb587d16 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:30.319303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90918161-5a18-40f1-8f73-967898e07333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:30.319957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90918161-5a18-40f1-8f73-967898e07333 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.320241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90918161-5a18-40f1-8f73-967898e07333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.320520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90918161-5a18-40f1-8f73-967898e07333 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.321285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90918161-5a18-40f1-8f73-967898e07333 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:03:30.321591 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90918161-5a18-40f1-8f73-967898e07333 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:30.323186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90918161-5a18-40f1-8f73-967898e07333 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:30.334901 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1335/2736] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:03:30.339301 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcbbeb-ea55-45c8-9c0f-d73b7b5ffc0f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:30.348615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcbbeb-ea55-45c8-9c0f-d73b7b5ffc0f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:30.353400 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdfc28-11ca-4ce4-af98-cca6bf2a40c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:30.354057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdfc28-11ca-4ce4-af98-cca6bf2a40c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.354347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdfc28-11ca-4ce4-af98-cca6bf2a40c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.354641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdfc28-11ca-4ce4-af98-cca6bf2a40c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.355834 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdfc28-11ca-4ce4-af98-cca6bf2a40c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:30.357771 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdfc28-11ca-4ce4-af98-cca6bf2a40c4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:30.365989 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:30.386250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1402/2737] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:30 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:30.390630 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90918161-5a18-40f1-8f73-967898e07333 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:30.400652 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90918161-5a18-40f1-8f73-967898e07333 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:30.440758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:30.458736 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f7ec6-1b39-4873-af34-015308d7d4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:30.459402 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f7ec6-1b39-4873-af34-015308d7d4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.459685 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f7ec6-1b39-4873-af34-015308d7d4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.459982 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f7ec6-1b39-4873-af34-015308d7d4e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.460699 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f7ec6-1b39-4873-af34-015308d7d4e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:30.462926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f7ec6-1b39-4873-af34-015308d7d4e8 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:30.485564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1336/2738] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:30 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:30.489958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdfc28-11ca-4ce4-af98-cca6bf2a40c4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:30.501101 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdfc28-11ca-4ce4-af98-cca6bf2a40c4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:30.505728 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9b7c1-49bb-448a-b493-4a44dcaf6b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:30.506386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9b7c1-49bb-448a-b493-4a44dcaf6b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.506676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9b7c1-49bb-448a-b493-4a44dcaf6b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.506959 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9b7c1-49bb-448a-b493-4a44dcaf6b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.507698 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9b7c1-49bb-448a-b493-4a44dcaf6b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:30.509335 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9b7c1-49bb-448a-b493-4a44dcaf6b58 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:30.520298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1403/2739] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:03:30.524357 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f7ec6-1b39-4873-af34-015308d7d4e8 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:30.534453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f7ec6-1b39-4873-af34-015308d7d4e8 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:30.539114 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8171e61-9fb7-4e78-b483-b264ca82700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:30.539761 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8171e61-9fb7-4e78-b483-b264ca82700d admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.540049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8171e61-9fb7-4e78-b483-b264ca82700d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.540331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8171e61-9fb7-4e78-b483-b264ca82700d admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.541088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8171e61-9fb7-4e78-b483-b264ca82700d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:30.543187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8171e61-9fb7-4e78-b483-b264ca82700d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:30.553903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1337/2740] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:30.558998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9b7c1-49bb-448a-b493-4a44dcaf6b58 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:30.569136 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9b7c1-49bb-448a-b493-4a44dcaf6b58 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:30.573780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735a3e9-005f-41a8-8f4d-a0fa12d19231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:30.574651 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735a3e9-005f-41a8-8f4d-a0fa12d19231 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.574937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735a3e9-005f-41a8-8f4d-a0fa12d19231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.575217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735a3e9-005f-41a8-8f4d-a0fa12d19231 admin admin] PATH_INFO: `/v3/projects/9512e5283c334e2281d77922b8c44820/users/29e50af0472b43aaa2ff968c3e274bd9/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.590353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735a3e9-005f-41a8-8f4d-a0fa12d19231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9512e5283c334e2281d77922b8c44820, user_id=29e50af0472b43aaa2ff968c3e274bd9, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:30.593075 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735a3e9-005f-41a8-8f4d-a0fa12d19231 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:30.606364 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1404/2741] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:30 2022] PUT /identity/v3/projects/9512e5283c334e2281d77922b8c44820/users/29e50af0472b43aaa2ff968c3e274bd9/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:30.618213 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2019f2b-877e-445e-831c-4f08455f13b1 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.618506 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2019f2b-877e-445e-831c-4f08455f13b1 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.618836 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2019f2b-877e-445e-831c-4f08455f13b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.646355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-d2019f2b-877e-445e-831c-4f08455f13b1 None None] MFA Rules not processed for user `29e50af0472b43aaa2ff968c3e274bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:30.647585 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-d2019f2b-877e-445e-831c-4f08455f13b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:30.712360 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1338/2742] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:30.717550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735a3e9-005f-41a8-8f4d-a0fa12d19231 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:30.779636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735a3e9-005f-41a8-8f4d-a0fa12d19231 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:30.784378 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee86ad8-a76e-42f6-bf72-3719280a11c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:30.785066 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee86ad8-a76e-42f6-bf72-3719280a11c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.785357 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee86ad8-a76e-42f6-bf72-3719280a11c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.785639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee86ad8-a76e-42f6-bf72-3719280a11c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.786342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee86ad8-a76e-42f6-bf72-3719280a11c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:30.788242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee86ad8-a76e-42f6-bf72-3719280a11c9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:30.814846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1405/2743] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:30 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:30.818774 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2019f2b-877e-445e-831c-4f08455f13b1 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:30.829484 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2019f2b-877e-445e-831c-4f08455f13b1 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:30.885166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faed4d-041b-4896-8058-d61226c9c54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:30.885830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faed4d-041b-4896-8058-d61226c9c54d admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.886114 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faed4d-041b-4896-8058-d61226c9c54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.886398 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faed4d-041b-4896-8058-d61226c9c54d admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.887088 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24faed4d-041b-4896-8058-d61226c9c54d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:30.889039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24faed4d-041b-4896-8058-d61226c9c54d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:30.908993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1339/2744] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:30 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:30.913069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee86ad8-a76e-42f6-bf72-3719280a11c9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:30.924537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee86ad8-a76e-42f6-bf72-3719280a11c9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:30.929486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db453723-3e38-4712-9f90-804b22cfdb09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:30.930133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db453723-3e38-4712-9f90-804b22cfdb09 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.930422 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db453723-3e38-4712-9f90-804b22cfdb09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.930706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db453723-3e38-4712-9f90-804b22cfdb09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.931437 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db453723-3e38-4712-9f90-804b22cfdb09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:30.933071 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db453723-3e38-4712-9f90-804b22cfdb09 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:30.943543 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1406/2745] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:30.947829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faed4d-041b-4896-8058-d61226c9c54d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:30.957903 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faed4d-041b-4896-8058-d61226c9c54d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:30.962631 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf08ed-f085-4b01-ad16-33faa766b06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:30.963511 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf08ed-f085-4b01-ad16-33faa766b06d admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:30.963801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf08ed-f085-4b01-ad16-33faa766b06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:30.964094 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf08ed-f085-4b01-ad16-33faa766b06d admin admin] PATH_INFO: `/v3/projects/dbf4caeeac114aa4b4a13ea07ddb2a1d/users/589db10e33d7484eaa249fb83f4365b7/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:30.982798 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf08ed-f085-4b01-ad16-33faa766b06d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf4caeeac114aa4b4a13ea07ddb2a1d, user_id=589db10e33d7484eaa249fb83f4365b7, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:30.985499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf08ed-f085-4b01-ad16-33faa766b06d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:30.998460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1340/2746] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:30 2022] PUT /identity/v3/projects/dbf4caeeac114aa4b4a13ea07ddb2a1d/users/589db10e33d7484eaa249fb83f4365b7/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:03:31.002372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db453723-3e38-4712-9f90-804b22cfdb09 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:31.012563 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db453723-3e38-4712-9f90-804b22cfdb09 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:31.069645 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fbda75-e978-456f-827f-d602df27dbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:31.070324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbda75-e978-456f-827f-d602df27dbcd admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:31.070617 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbda75-e978-456f-827f-d602df27dbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:31.070899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fbda75-e978-456f-827f-d602df27dbcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:31.071636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbda75-e978-456f-827f-d602df27dbcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:31.073298 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fbda75-e978-456f-827f-d602df27dbcd admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:31.084873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1407/2747] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:31.089048 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf08ed-f085-4b01-ad16-33faa766b06d admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:31.099940 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf08ed-f085-4b01-ad16-33faa766b06d admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:31.104480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfd0fb-6217-46d4-a594-7ab1fa8805aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:31.105388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfd0fb-6217-46d4-a594-7ab1fa8805aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:31.105676 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfd0fb-6217-46d4-a594-7ab1fa8805aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:31.105955 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfd0fb-6217-46d4-a594-7ab1fa8805aa admin admin] PATH_INFO: `/v3/projects/dbf4caeeac114aa4b4a13ea07ddb2a1d/users/589db10e33d7484eaa249fb83f4365b7/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:31.108027 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfd0fb-6217-46d4-a594-7ab1fa8805aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf4caeeac114aa4b4a13ea07ddb2a1d, user_id=589db10e33d7484eaa249fb83f4365b7, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:31.110753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfd0fb-6217-46d4-a594-7ab1fa8805aa admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:31.121169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1341/2748] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:31 2022] PUT /identity/v3/projects/dbf4caeeac114aa4b4a13ea07ddb2a1d/users/589db10e33d7484eaa249fb83f4365b7/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:31.131092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1546fa-6885-4886-a481-883cef5a7b5e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:31.131391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1546fa-6885-4886-a481-883cef5a7b5e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:31.131679 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1546fa-6885-4886-a481-883cef5a7b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:31.144472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfd0fb-6217-46d4-a594-7ab1fa8805aa admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:31.146807 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a5cfd0fb-6217-46d4-a594-7ab1fa8805aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:31.152911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:31.168705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-ad1546fa-6885-4886-a481-883cef5a7b5e None None] MFA Rules not processed for user `589db10e33d7484eaa249fb83f4365b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:31.169942 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ad1546fa-6885-4886-a481-883cef5a7b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:31.182578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:31.221958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfd0fb-6217-46d4-a594-7ab1fa8805aa admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:31.227226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a54d9-2106-4f21-9a38-0f33a713c1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:31.228130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a54d9-2106-4f21-9a38-0f33a713c1af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:31.228515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a54d9-2106-4f21-9a38-0f33a713c1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:31.228737 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a54d9-2106-4f21-9a38-0f33a713c1af admin admin] PATH_INFO: `/v3/users/07fbea52894e4707bbb1d4f15b42a80e` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:31.230096 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a54d9-2106-4f21-9a38-0f33a713c1af admin admin] RBAC: Authorizing `identity:delete_user(user_id=07fbea52894e4707bbb1d4f15b42a80e)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:31.232074 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a54d9-2106-4f21-9a38-0f33a713c1af admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:31.285087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bb0a54d9-2106-4f21-9a38-0f33a713c1af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07fbea52894e4707bbb1d4f15b42a80e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:31.285786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bb0a54d9-2106-4f21-9a38-0f33a713c1af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07fbea52894e4707bbb1d4f15b42a80e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:31.286420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bb0a54d9-2106-4f21-9a38-0f33a713c1af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07fbea52894e4707bbb1d4f15b42a80e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:31.299806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bb0a54d9-2106-4f21-9a38-0f33a713c1af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07fbea52894e4707bbb1d4f15b42a80e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:31.307186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-bb0a54d9-2106-4f21-9a38-0f33a713c1af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07fbea52894e4707bbb1d4f15b42a80e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:31.320878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1342/2749] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:31 2022] DELETE /identity/v3/users/07fbea52894e4707bbb1d4f15b42a80e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:03:31.326315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38403896-dede-49b6-b3ce-7f42e835056d None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:31.326608 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38403896-dede-49b6-b3ce-7f42e835056d None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:31.326889 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38403896-dede-49b6-b3ce-7f42e835056d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:31.345269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-38403896-dede-49b6-b3ce-7f42e835056d None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:31.346503 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-38403896-dede-49b6-b3ce-7f42e835056d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:31.368976 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1408/2750] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:31 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:31.380691 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1546fa-6885-4886-a481-883cef5a7b5e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:31.382070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ad1546fa-6885-4886-a481-883cef5a7b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:31.409498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1343/2751] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:31.418973 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38403896-dede-49b6-b3ce-7f42e835056d None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:31.420388 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-38403896-dede-49b6-b3ce-7f42e835056d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:31.440231 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1546fa-6885-4886-a481-883cef5a7b5e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:31.448444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:31.483061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38403896-dede-49b6-b3ce-7f42e835056d None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:31.486932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf1770-1f1e-4a37-befd-3c164ecba85c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:31.487610 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf1770-1f1e-4a37-befd-3c164ecba85c service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:31.487894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf1770-1f1e-4a37-befd-3c164ecba85c service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:31.488175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf1770-1f1e-4a37-befd-3c164ecba85c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:31.490889 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800e9ae-b3ae-4a79-94c5-bac5f88ead0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:31.491541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e9ae-b3ae-4a79-94c5-bac5f88ead0e service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:31.491832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e9ae-b3ae-4a79-94c5-bac5f88ead0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:31.492116 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e9ae-b3ae-4a79-94c5-bac5f88ead0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:31.498280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf1770-1f1e-4a37-befd-3c164ecba85c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:31.499718 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf1770-1f1e-4a37-befd-3c164ecba85c service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:31.509444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1344/2752] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:03:31.553700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e9ae-b3ae-4a79-94c5-bac5f88ead0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:31.555135 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e9ae-b3ae-4a79-94c5-bac5f88ead0e service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:31.566082 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1409/2753] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:31 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:03:31.638913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf1770-1f1e-4a37-befd-3c164ecba85c service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:31.648405 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf1770-1f1e-4a37-befd-3c164ecba85c service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:31.652719 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f1df8-c581-4941-8e30-49dcc84f0846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:31.653427 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f1df8-c581-4941-8e30-49dcc84f0846 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:31.653898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f1df8-c581-4941-8e30-49dcc84f0846 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:31.654242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f1df8-c581-4941-8e30-49dcc84f0846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:31.659571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-193f1df8-c581-4941-8e30-49dcc84f0846 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:31.731079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f1df8-c581-4941-8e30-49dcc84f0846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:31.732525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f1df8-c581-4941-8e30-49dcc84f0846 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:31.742178 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1345/2754] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:31.794566 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800e9ae-b3ae-4a79-94c5-bac5f88ead0e service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:31.796089 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-2800e9ae-b3ae-4a79-94c5-bac5f88ead0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:31.811350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800e9ae-b3ae-4a79-94c5-bac5f88ead0e service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:31.816656 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372904e-85fb-4dd3-a50a-45ba69e01254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:31.817595 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372904e-85fb-4dd3-a50a-45ba69e01254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:31.817878 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372904e-85fb-4dd3-a50a-45ba69e01254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:31.818162 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372904e-85fb-4dd3-a50a-45ba69e01254 admin admin] PATH_INFO: `/v3/projects/75e59db7e77d41bf9c46aa7c010b41b0` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:31.819371 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372904e-85fb-4dd3-a50a-45ba69e01254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75e59db7e77d41bf9c46aa7c010b41b0)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:31.821507 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372904e-85fb-4dd3-a50a-45ba69e01254 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:31.833145 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=100 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:31.881782 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b372904e-85fb-4dd3-a50a-45ba69e01254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75e59db7e77d41bf9c46aa7c010b41b0', 'request_id': 'req-b372904e-85fb-4dd3-a50a-45ba69e01254'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:31.891919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b372904e-85fb-4dd3-a50a-45ba69e01254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75e59db7e77d41bf9c46aa7c010b41b0', 'request_id': 'req-b372904e-85fb-4dd3-a50a-45ba69e01254'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:31.900349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b372904e-85fb-4dd3-a50a-45ba69e01254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75e59db7e77d41bf9c46aa7c010b41b0', 'request_id': 'req-b372904e-85fb-4dd3-a50a-45ba69e01254'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:31.911918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b372904e-85fb-4dd3-a50a-45ba69e01254 admin admin] The token cache is being invalidate because project 75e59db7e77d41bf9c46aa7c010b41b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:31.912207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b372904e-85fb-4dd3-a50a-45ba69e01254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:31.912895 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-b372904e-85fb-4dd3-a50a-45ba69e01254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:31.914475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1410/2755] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:31 2022] DELETE /identity/v3/projects/75e59db7e77d41bf9c46aa7c010b41b0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:31.919921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfa457-096a-4073-88b6-56bf206d2900 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:31.920219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfa457-096a-4073-88b6-56bf206d2900 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:31.920508 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfa457-096a-4073-88b6-56bf206d2900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:31.938756 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-95cfa457-096a-4073-88b6-56bf206d2900 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:31.940079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-95cfa457-096a-4073-88b6-56bf206d2900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:31.999698 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1346/2756] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:03:32.012457 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d13378-a421-4aeb-80c7-e9d15227ea5f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.012785 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d13378-a421-4aeb-80c7-e9d15227ea5f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.013069 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d13378-a421-4aeb-80c7-e9d15227ea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.031951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-b2d13378-a421-4aeb-80c7-e9d15227ea5f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:32.033217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b2d13378-a421-4aeb-80c7-e9d15227ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:32.041216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1411/2757] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:32.045887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfa457-096a-4073-88b6-56bf206d2900 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:32.055935 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfa457-096a-4073-88b6-56bf206d2900 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:32.061264 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4089a-16a8-4e2d-89fe-c2302ee36d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:32.061916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4089a-16a8-4e2d-89fe-c2302ee36d22 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.062200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4089a-16a8-4e2d-89fe-c2302ee36d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.062485 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4089a-16a8-4e2d-89fe-c2302ee36d22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.063214 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4089a-16a8-4e2d-89fe-c2302ee36d22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:03:32.063523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4089a-16a8-4e2d-89fe-c2302ee36d22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:32.065161 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4089a-16a8-4e2d-89fe-c2302ee36d22 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:32.076768 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1347/2758] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:03:32.081009 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d13378-a421-4aeb-80c7-e9d15227ea5f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:32.090390 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d13378-a421-4aeb-80c7-e9d15227ea5f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:32.095055 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e0301-4dc2-4ae3-920c-7fa501dd8fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:32.095761 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e0301-4dc2-4ae3-920c-7fa501dd8fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.096045 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e0301-4dc2-4ae3-920c-7fa501dd8fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.096324 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e0301-4dc2-4ae3-920c-7fa501dd8fbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.097074 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e0301-4dc2-4ae3-920c-7fa501dd8fbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:32.098982 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e0301-4dc2-4ae3-920c-7fa501dd8fbc admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:32.133070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1412/2759] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:32 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:03:32.137318 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4089a-16a8-4e2d-89fe-c2302ee36d22 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:32.147472 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4089a-16a8-4e2d-89fe-c2302ee36d22 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:32.195522 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=100 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:32.206332 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e912c-040e-4db1-9830-bf60b956f9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:32.207000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e912c-040e-4db1-9830-bf60b956f9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.207289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e912c-040e-4db1-9830-bf60b956f9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.207569 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e912c-040e-4db1-9830-bf60b956f9b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.208258 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e912c-040e-4db1-9830-bf60b956f9b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:32.210261 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e912c-040e-4db1-9830-bf60b956f9b4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:32.231728 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1348/2760] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:32 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:32.235861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e0301-4dc2-4ae3-920c-7fa501dd8fbc admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:32.249153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e0301-4dc2-4ae3-920c-7fa501dd8fbc admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:32.253875 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ea77d-6d17-4106-9711-9a6eb2e66d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:32.254477 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ea77d-6d17-4106-9711-9a6eb2e66d7e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.254760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ea77d-6d17-4106-9711-9a6eb2e66d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.255036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351ea77d-6d17-4106-9711-9a6eb2e66d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.255786 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ea77d-6d17-4106-9711-9a6eb2e66d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:32.257391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351ea77d-6d17-4106-9711-9a6eb2e66d7e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:32.268124 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1413/2761] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:32.272298 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e912c-040e-4db1-9830-bf60b956f9b4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:32.282953 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e912c-040e-4db1-9830-bf60b956f9b4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:32.287594 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82176d6c-7a45-4ab7-9d1a-04dd8044d71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:32.288257 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82176d6c-7a45-4ab7-9d1a-04dd8044d71c admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.288544 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82176d6c-7a45-4ab7-9d1a-04dd8044d71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.288860 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82176d6c-7a45-4ab7-9d1a-04dd8044d71c admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.289598 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82176d6c-7a45-4ab7-9d1a-04dd8044d71c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:32.291198 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82176d6c-7a45-4ab7-9d1a-04dd8044d71c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:32.303804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1349/2762] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:32.308006 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ea77d-6d17-4106-9711-9a6eb2e66d7e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:32.318061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351ea77d-6d17-4106-9711-9a6eb2e66d7e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:32.322820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb16f3-172a-4115-b04b-3d004dbbccba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:32.323847 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb16f3-172a-4115-b04b-3d004dbbccba admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.324130 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb16f3-172a-4115-b04b-3d004dbbccba admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.324411 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb16f3-172a-4115-b04b-3d004dbbccba admin admin] PATH_INFO: `/v3/projects/26766a57e1e64b9789ec4f5a8721dbdf/users/564830226cb84a2baab52a43898343a7/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.339926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb16f3-172a-4115-b04b-3d004dbbccba admin admin] RBAC: Authorizing `identity:create_grant(project_id=26766a57e1e64b9789ec4f5a8721dbdf, user_id=564830226cb84a2baab52a43898343a7, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:32.342653 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb16f3-172a-4115-b04b-3d004dbbccba admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:32.356130 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1414/2763] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:32 2022] PUT /identity/v3/projects/26766a57e1e64b9789ec4f5a8721dbdf/users/564830226cb84a2baab52a43898343a7/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:32.370458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a30a3-3517-472b-b420-a525baba5cd3 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.370758 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a30a3-3517-472b-b420-a525baba5cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.371040 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a30a3-3517-472b-b420-a525baba5cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.400033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-b32a30a3-3517-472b-b420-a525baba5cd3 None None] MFA Rules not processed for user `564830226cb84a2baab52a43898343a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:32.401311 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b32a30a3-3517-472b-b420-a525baba5cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:32.467109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1350/2764] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:32.471556 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb16f3-172a-4115-b04b-3d004dbbccba admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:32.534299 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb16f3-172a-4115-b04b-3d004dbbccba admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:32.539036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056a966-e143-4643-8b20-111672adea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:32.539700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056a966-e143-4643-8b20-111672adea55 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.539991 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056a966-e143-4643-8b20-111672adea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.540278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056a966-e143-4643-8b20-111672adea55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.541020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056a966-e143-4643-8b20-111672adea55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:32.543635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056a966-e143-4643-8b20-111672adea55 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:32.570448 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1415/2765] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:32 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:32.575149 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a30a3-3517-472b-b420-a525baba5cd3 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:32.585844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a30a3-3517-472b-b420-a525baba5cd3 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:32.641835 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b878dd6-5f0f-45d1-904c-a82a883e6f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:32.642692 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b878dd6-5f0f-45d1-904c-a82a883e6f3e admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.642799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b878dd6-5f0f-45d1-904c-a82a883e6f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.643069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b878dd6-5f0f-45d1-904c-a82a883e6f3e admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.643754 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b878dd6-5f0f-45d1-904c-a82a883e6f3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:32.645710 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b878dd6-5f0f-45d1-904c-a82a883e6f3e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:32.669499 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1351/2766] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:32 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:32.673622 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056a966-e143-4643-8b20-111672adea55 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:32.685349 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056a966-e143-4643-8b20-111672adea55 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:32.690240 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2635d7-319d-4a77-bc5c-3cddce93fbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:32.690926 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2635d7-319d-4a77-bc5c-3cddce93fbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.691214 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2635d7-319d-4a77-bc5c-3cddce93fbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.691496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2635d7-319d-4a77-bc5c-3cddce93fbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.692244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2635d7-319d-4a77-bc5c-3cddce93fbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:32.693958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2635d7-319d-4a77-bc5c-3cddce93fbd2 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:32.705530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1416/2767] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:32.709831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b878dd6-5f0f-45d1-904c-a82a883e6f3e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:32.721113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b878dd6-5f0f-45d1-904c-a82a883e6f3e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:32.727256 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12004e-32ee-4942-be30-ea7b49efeff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:32.727469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12004e-32ee-4942-be30-ea7b49efeff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.727652 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12004e-32ee-4942-be30-ea7b49efeff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.727939 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12004e-32ee-4942-be30-ea7b49efeff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.728726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12004e-32ee-4942-be30-ea7b49efeff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:32.730340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12004e-32ee-4942-be30-ea7b49efeff2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:32.742717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1352/2768] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:03:32.747231 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2635d7-319d-4a77-bc5c-3cddce93fbd2 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:32.757647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2635d7-319d-4a77-bc5c-3cddce93fbd2 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:32.762912 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd2cb2-2a26-414b-8e87-70570f059829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:32.763797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd2cb2-2a26-414b-8e87-70570f059829 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.764091 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd2cb2-2a26-414b-8e87-70570f059829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.764364 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd2cb2-2a26-414b-8e87-70570f059829 admin admin] PATH_INFO: `/v3/projects/b285de93e2b949eabbd13ccbe04379f3/users/0abbb04725ec40dda9ad5eb940d669a1/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.767565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:32.783269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd2cb2-2a26-414b-8e87-70570f059829 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b285de93e2b949eabbd13ccbe04379f3, user_id=0abbb04725ec40dda9ad5eb940d669a1, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:32.786010 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd2cb2-2a26-414b-8e87-70570f059829 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:32.798951 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1417/2769] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:32 2022] PUT /identity/v3/projects/b285de93e2b949eabbd13ccbe04379f3/users/0abbb04725ec40dda9ad5eb940d669a1/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:32.809497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c919e84-65d8-4090-a6db-aeccc54add47 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.809799 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c919e84-65d8-4090-a6db-aeccc54add47 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.810082 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c919e84-65d8-4090-a6db-aeccc54add47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.838528 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-7c919e84-65d8-4090-a6db-aeccc54add47 None None] MFA Rules not processed for user `0abbb04725ec40dda9ad5eb940d669a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:32.840392 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7c919e84-65d8-4090-a6db-aeccc54add47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:32.867337 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:32.909949 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1353/2770] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:32.914603 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd2cb2-2a26-414b-8e87-70570f059829 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:32.980814 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd2cb2-2a26-414b-8e87-70570f059829 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:32.985541 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57643570-acbf-412b-a100-8bf3881fba10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:32.986201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57643570-acbf-412b-a100-8bf3881fba10 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:32.986489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57643570-acbf-412b-a100-8bf3881fba10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:32.986773 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57643570-acbf-412b-a100-8bf3881fba10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:32.987465 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57643570-acbf-412b-a100-8bf3881fba10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:32.989391 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57643570-acbf-412b-a100-8bf3881fba10 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:33.020435 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1418/2771] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:03:32 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:33.022125 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c919e84-65d8-4090-a6db-aeccc54add47 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:33.034038 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c919e84-65d8-4090-a6db-aeccc54add47 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:33.097171 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18ad0c-b5e5-4d08-9583-837292a81f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:33.097849 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18ad0c-b5e5-4d08-9583-837292a81f8e admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:33.098141 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18ad0c-b5e5-4d08-9583-837292a81f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:33.098431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a18ad0c-b5e5-4d08-9583-837292a81f8e admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:33.099103 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18ad0c-b5e5-4d08-9583-837292a81f8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:33.101046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a18ad0c-b5e5-4d08-9583-837292a81f8e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:33.124011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1354/2772] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:03:33 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:03:33.125481 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57643570-acbf-412b-a100-8bf3881fba10 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:33.137057 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57643570-acbf-412b-a100-8bf3881fba10 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:33.142262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b4085-bf31-47e8-a4b5-0da6c72a911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:33.142929 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4085-bf31-47e8-a4b5-0da6c72a911e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:33.143224 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4085-bf31-47e8-a4b5-0da6c72a911e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:33.143505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b4085-bf31-47e8-a4b5-0da6c72a911e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:33.144241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b4085-bf31-47e8-a4b5-0da6c72a911e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:33.145870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b4085-bf31-47e8-a4b5-0da6c72a911e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:33.159902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1419/2773] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:33.165623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18ad0c-b5e5-4d08-9583-837292a81f8e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:33.177648 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a18ad0c-b5e5-4d08-9583-837292a81f8e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:33.182579 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae248c-1028-41b8-960c-e71236dd03dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:33.183514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae248c-1028-41b8-960c-e71236dd03dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:33.183811 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae248c-1028-41b8-960c-e71236dd03dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:33.184085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae248c-1028-41b8-960c-e71236dd03dd admin admin] PATH_INFO: `/v3/projects/9c15762636e542a18ed62db67f701dc7/users/d1045bc6febd43f9a0c60e93e8e25c42/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:33.202404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae248c-1028-41b8-960c-e71236dd03dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c15762636e542a18ed62db67f701dc7, user_id=d1045bc6febd43f9a0c60e93e8e25c42, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:33.205275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae248c-1028-41b8-960c-e71236dd03dd admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:33.222978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1355/2774] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:33 2022] PUT /identity/v3/projects/9c15762636e542a18ed62db67f701dc7/users/d1045bc6febd43f9a0c60e93e8e25c42/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:33.227279 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b4085-bf31-47e8-a4b5-0da6c72a911e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:33.243612 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b4085-bf31-47e8-a4b5-0da6c72a911e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:33.321053 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7781f64-a6af-44a9-93d1-e9eef65ee70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:33.321745 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7781f64-a6af-44a9-93d1-e9eef65ee70e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:33.322020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7781f64-a6af-44a9-93d1-e9eef65ee70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:33.322293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7781f64-a6af-44a9-93d1-e9eef65ee70e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:33.323024 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7781f64-a6af-44a9-93d1-e9eef65ee70e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:33.325219 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7781f64-a6af-44a9-93d1-e9eef65ee70e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:33.335802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1420/2775] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:03:33.340785 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae248c-1028-41b8-960c-e71236dd03dd admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:33.352121 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae248c-1028-41b8-960c-e71236dd03dd admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:33.357172 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ffdf7-1dea-4b26-b2c7-05bc78473dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:33.358085 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ffdf7-1dea-4b26-b2c7-05bc78473dda admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:33.358369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ffdf7-1dea-4b26-b2c7-05bc78473dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:33.358652 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ffdf7-1dea-4b26-b2c7-05bc78473dda admin admin] PATH_INFO: `/v3/projects/9c15762636e542a18ed62db67f701dc7/users/d1045bc6febd43f9a0c60e93e8e25c42/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:33.360700 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ffdf7-1dea-4b26-b2c7-05bc78473dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c15762636e542a18ed62db67f701dc7, user_id=d1045bc6febd43f9a0c60e93e8e25c42, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:33.363368 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3ffdf7-1dea-4b26-b2c7-05bc78473dda admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:33.375439 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1356/2776] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:03:33 2022] PUT /identity/v3/projects/9c15762636e542a18ed62db67f701dc7/users/d1045bc6febd43f9a0c60e93e8e25c42/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:33.385760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf6cec-0e6a-48f9-a304-db08fbd76cdb None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:33.386552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf6cec-0e6a-48f9-a304-db08fbd76cdb None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:33.386836 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf6cec-0e6a-48f9-a304-db08fbd76cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:33.414236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-70cf6cec-0e6a-48f9-a304-db08fbd76cdb None None] MFA Rules not processed for user `d1045bc6febd43f9a0c60e93e8e25c42`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:33.415470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-70cf6cec-0e6a-48f9-a304-db08fbd76cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:33.458143 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:33.484194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1421/2777] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:33 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:33.495708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ffdf7-1dea-4b26-b2c7-05bc78473dda admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:33.497611 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4a3ffdf7-1dea-4b26-b2c7-05bc78473dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:33.558365 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ffdf7-1dea-4b26-b2c7-05bc78473dda admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:33.561844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa7e1b-d927-42b0-9121-a80497e3159b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:33.562477 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa7e1b-d927-42b0-9121-a80497e3159b service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:33.562763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa7e1b-d927-42b0-9121-a80497e3159b service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:33.563041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa7e1b-d927-42b0-9121-a80497e3159b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:33.597540 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:33.622616 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa7e1b-d927-42b0-9121-a80497e3159b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:33.624058 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa7e1b-d927-42b0-9121-a80497e3159b service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:33.632913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1357/2778] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:35.105649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf6cec-0e6a-48f9-a304-db08fbd76cdb None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:35.115494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cf6cec-0e6a-48f9-a304-db08fbd76cdb None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:35.119070 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4697bd-561b-49fd-9788-cbf4817333f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:35.119706 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4697bd-561b-49fd-9788-cbf4817333f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:35.119987 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4697bd-561b-49fd-9788-cbf4817333f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:35.120262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4697bd-561b-49fd-9788-cbf4817333f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:35.183767 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4697bd-561b-49fd-9788-cbf4817333f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:35.185217 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4697bd-561b-49fd-9788-cbf4817333f5 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:35.195255 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1422/2779] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:35.836645 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa7e1b-d927-42b0-9121-a80497e3159b service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:35.845359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa7e1b-d927-42b0-9121-a80497e3159b service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:35.849306 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86fff6-0247-4682-b2c4-d1333c519e04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:35.849946 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86fff6-0247-4682-b2c4-d1333c519e04 service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:35.850237 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86fff6-0247-4682-b2c4-d1333c519e04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:35.850525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86fff6-0247-4682-b2c4-d1333c519e04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:35.860629 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86fff6-0247-4682-b2c4-d1333c519e04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:35.862087 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86fff6-0247-4682-b2c4-d1333c519e04 service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:35.872362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1358/2780] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:40.806402 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4697bd-561b-49fd-9788-cbf4817333f5 service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:40.872852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4697bd-561b-49fd-9788-cbf4817333f5 service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:40.877755 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:40.878824 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:40.881049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:40.881342 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] PATH_INFO: `/v3/users/564830226cb84a2baab52a43898343a7` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:40.882612 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=564830226cb84a2baab52a43898343a7)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:40.884611 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:40.934143 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564830226cb84a2baab52a43898343a7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:40.934815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564830226cb84a2baab52a43898343a7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:40.935470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564830226cb84a2baab52a43898343a7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:40.948409 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564830226cb84a2baab52a43898343a7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:40.955845 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564830226cb84a2baab52a43898343a7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:40.969585 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1423/2781] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:40 2022] DELETE /identity/v3/users/564830226cb84a2baab52a43898343a7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:40.972768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:40.974236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44112753-cbe2-4859-8e3c-897eaf476d55 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:40.974713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44112753-cbe2-4859-8e3c-897eaf476d55 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:40.974837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44112753-cbe2-4859-8e3c-897eaf476d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:40.992659 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-44112753-cbe2-4859-8e3c-897eaf476d55 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:03:40.993910 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-44112753-cbe2-4859-8e3c-897eaf476d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:41.059841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1359/2782] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:03:41.072348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:41.072574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:41.130250 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee298d9-3dfe-441d-b4f1-1ad8c96fea7f admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:41.133940 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61620acb-93c1-440c-b0eb-db411150be6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:41.134579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61620acb-93c1-440c-b0eb-db411150be6c service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:41.134869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61620acb-93c1-440c-b0eb-db411150be6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:41.135151 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61620acb-93c1-440c-b0eb-db411150be6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:41.144080 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61620acb-93c1-440c-b0eb-db411150be6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:41.145537 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61620acb-93c1-440c-b0eb-db411150be6c service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:41.155338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1424/2783] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:03:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:03:41.433850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44112753-cbe2-4859-8e3c-897eaf476d55 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:41.435300 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-44112753-cbe2-4859-8e3c-897eaf476d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:41.455041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44112753-cbe2-4859-8e3c-897eaf476d55 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:41.460633 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:41.461478 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:41.461760 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:41.462068 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] PATH_INFO: `/v3/projects/26766a57e1e64b9789ec4f5a8721dbdf` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:41.463368 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26766a57e1e64b9789ec4f5a8721dbdf)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:41.465479 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:41.511242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26766a57e1e64b9789ec4f5a8721dbdf', 'request_id': 'req-e99e01dc-0536-4950-993a-2cb92e8f1f98'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:41.519498 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26766a57e1e64b9789ec4f5a8721dbdf', 'request_id': 'req-e99e01dc-0536-4950-993a-2cb92e8f1f98'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:41.526117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26766a57e1e64b9789ec4f5a8721dbdf', 'request_id': 'req-e99e01dc-0536-4950-993a-2cb92e8f1f98'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:41.536318 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] The token cache is being invalidate because project 26766a57e1e64b9789ec4f5a8721dbdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:41.536662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:41.537329 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:41.538859 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1360/2784] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:41 2022] DELETE /identity/v3/projects/26766a57e1e64b9789ec4f5a8721dbdf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:41.543485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61620acb-93c1-440c-b0eb-db411150be6c service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:41.553469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61620acb-93c1-440c-b0eb-db411150be6c service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:41.609419 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:41.610293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:41.610580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:41.610859 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] PATH_INFO: `/v3/users/0abbb04725ec40dda9ad5eb940d669a1` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:41.612050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0abbb04725ec40dda9ad5eb940d669a1)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:41.614002 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:41.660173 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0abbb04725ec40dda9ad5eb940d669a1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:41.660887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0abbb04725ec40dda9ad5eb940d669a1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:41.661516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0abbb04725ec40dda9ad5eb940d669a1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:41.674555 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0abbb04725ec40dda9ad5eb940d669a1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:41.681483 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0abbb04725ec40dda9ad5eb940d669a1'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:41.695873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1425/2785] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:41 2022] DELETE /identity/v3/users/0abbb04725ec40dda9ad5eb940d669a1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:41.969676 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:41.971207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:41.999355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e01dc-0536-4950-993a-2cb92e8f1f98 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:42.048424 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:42.065847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:42.066717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:42.067002 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:42.067404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] PATH_INFO: `/v3/projects/b285de93e2b949eabbd13ccbe04379f3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:42.068482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b285de93e2b949eabbd13ccbe04379f3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:42.070476 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:42.117654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b285de93e2b949eabbd13ccbe04379f3', 'request_id': 'req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.126445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b285de93e2b949eabbd13ccbe04379f3', 'request_id': 'req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.136886 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b285de93e2b949eabbd13ccbe04379f3', 'request_id': 'req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.147596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] The token cache is being invalidate because project b285de93e2b949eabbd13ccbe04379f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:42.147897 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.148555 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.150219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1361/2786] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:41 2022] DELETE /identity/v3/projects/b285de93e2b949eabbd13ccbe04379f3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:42.154802 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:42.156229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:42.170676 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a14697-be9e-4c13-82e5-c93be6b26e4e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:42.187922 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:42.238429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:42.239307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:42.239589 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:42.239931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] PATH_INFO: `/v3/users/d1045bc6febd43f9a0c60e93e8e25c42` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:42.241196 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1045bc6febd43f9a0c60e93e8e25c42)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:42.243099 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:42.291552 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1045bc6febd43f9a0c60e93e8e25c42'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.292218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1045bc6febd43f9a0c60e93e8e25c42'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.293470 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1045bc6febd43f9a0c60e93e8e25c42'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.307322 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1045bc6febd43f9a0c60e93e8e25c42'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.315705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1045bc6febd43f9a0c60e93e8e25c42'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.328961 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1426/2787] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:03:42 2022] DELETE /identity/v3/users/d1045bc6febd43f9a0c60e93e8e25c42 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:42.611155 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:03:42.612726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:03:42.627140 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ec7963-25f5-4bc4-9eae-279b2fc8aa27 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:03:42.687978 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ef369-a2f6-4d68-8a23-3ce7af157bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:03:42.688870 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ef369-a2f6-4d68-8a23-3ce7af157bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:03:42.689165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ef369-a2f6-4d68-8a23-3ce7af157bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:03:42.689445 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ef369-a2f6-4d68-8a23-3ce7af157bae admin admin] PATH_INFO: `/v3/projects/9c15762636e542a18ed62db67f701dc7` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:03:42.690686 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ef369-a2f6-4d68-8a23-3ce7af157bae admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c15762636e542a18ed62db67f701dc7)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:03:42.692932 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ef369-a2f6-4d68-8a23-3ce7af157bae admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:03:42.738433 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-da3ef369-a2f6-4d68-8a23-3ce7af157bae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c15762636e542a18ed62db67f701dc7', 'request_id': 'req-da3ef369-a2f6-4d68-8a23-3ce7af157bae'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.745448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-da3ef369-a2f6-4d68-8a23-3ce7af157bae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c15762636e542a18ed62db67f701dc7', 'request_id': 'req-da3ef369-a2f6-4d68-8a23-3ce7af157bae'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.751958 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-da3ef369-a2f6-4d68-8a23-3ce7af157bae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c15762636e542a18ed62db67f701dc7', 'request_id': 'req-da3ef369-a2f6-4d68-8a23-3ce7af157bae'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.762756 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-da3ef369-a2f6-4d68-8a23-3ce7af157bae admin admin] The token cache is being invalidate because project 9c15762636e542a18ed62db67f701dc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:03:42.763039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-da3ef369-a2f6-4d68-8a23-3ce7af157bae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.763668 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-da3ef369-a2f6-4d68-8a23-3ce7af157bae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:03:42.765259 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1362/2788] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:03:42 2022] DELETE /identity/v3/projects/9c15762636e542a18ed62db67f701dc7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:03:52.325262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=44,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:03:52.759076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:09.885821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:09.887280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:09.900282 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c31a02-5ef2-43fe-99fa-8a66fdd2bb97 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:09.955170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:09.956058 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:09.956348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:09.956629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] PATH_INFO: `/v3/users/29e50af0472b43aaa2ff968c3e274bd9` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:09.957855 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] RBAC: Authorizing `identity:delete_user(user_id=29e50af0472b43aaa2ff968c3e274bd9)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:09.959765 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:10.005986 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29e50af0472b43aaa2ff968c3e274bd9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.006642 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29e50af0472b43aaa2ff968c3e274bd9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.007256 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29e50af0472b43aaa2ff968c3e274bd9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.019033 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29e50af0472b43aaa2ff968c3e274bd9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.025870 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29e50af0472b43aaa2ff968c3e274bd9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.038654 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1427/2789] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:04:09 2022] DELETE /identity/v3/users/29e50af0472b43aaa2ff968c3e274bd9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:04:10.043184 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76227ddf-8b58-4b14-9aa1-39991ca1399c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:10.043711 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76227ddf-8b58-4b14-9aa1-39991ca1399c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:10.043837 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76227ddf-8b58-4b14-9aa1-39991ca1399c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:10.062173 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-76227ddf-8b58-4b14-9aa1-39991ca1399c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:10.063406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-76227ddf-8b58-4b14-9aa1-39991ca1399c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:10.121280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1363/2790] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:04:10.130918 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:10.132308 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:10.192284 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32b198-febe-4d7b-a4d7-748b53f7700d admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:10.198520 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492582ca-2c01-4645-a493-623903553896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:10.199231 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492582ca-2c01-4645-a493-623903553896 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:10.199540 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492582ca-2c01-4645-a493-623903553896 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:10.199821 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492582ca-2c01-4645-a493-623903553896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:10.210847 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492582ca-2c01-4645-a493-623903553896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:10.212319 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492582ca-2c01-4645-a493-623903553896 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:10.222329 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1428/2791] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:04:10.492113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76227ddf-8b58-4b14-9aa1-39991ca1399c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:10.493664 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-76227ddf-8b58-4b14-9aa1-39991ca1399c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:10.505706 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76227ddf-8b58-4b14-9aa1-39991ca1399c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:10.510947 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:10.511804 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:10.512084 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:10.512362 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] PATH_INFO: `/v3/projects/9512e5283c334e2281d77922b8c44820` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:10.513614 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9512e5283c334e2281d77922b8c44820)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:10.515511 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:10.569092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9512e5283c334e2281d77922b8c44820', 'request_id': 'req-f861ff7c-6683-47d5-9074-3481740969b1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.576579 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9512e5283c334e2281d77922b8c44820', 'request_id': 'req-f861ff7c-6683-47d5-9074-3481740969b1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.583505 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9512e5283c334e2281d77922b8c44820', 'request_id': 'req-f861ff7c-6683-47d5-9074-3481740969b1'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.594059 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] The token cache is being invalidate because project 9512e5283c334e2281d77922b8c44820 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:04:10.594355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.595004 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.597130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1364/2792] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:04:10 2022] DELETE /identity/v3/projects/9512e5283c334e2281d77922b8c44820 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:10.601181 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492582ca-2c01-4645-a493-623903553896 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:10.610900 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492582ca-2c01-4645-a493-623903553896 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:10.628830 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:10.670091 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73658558-850e-45c2-9619-c95a05dab270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:10.671072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73658558-850e-45c2-9619-c95a05dab270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:10.671362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73658558-850e-45c2-9619-c95a05dab270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:10.671639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73658558-850e-45c2-9619-c95a05dab270 admin admin] PATH_INFO: `/v3/users/589db10e33d7484eaa249fb83f4365b7` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:10.672883 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73658558-850e-45c2-9619-c95a05dab270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=589db10e33d7484eaa249fb83f4365b7)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:10.674820 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73658558-850e-45c2-9619-c95a05dab270 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:10.720209 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-73658558-850e-45c2-9619-c95a05dab270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '589db10e33d7484eaa249fb83f4365b7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.720921 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-73658558-850e-45c2-9619-c95a05dab270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '589db10e33d7484eaa249fb83f4365b7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.721532 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-73658558-850e-45c2-9619-c95a05dab270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '589db10e33d7484eaa249fb83f4365b7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.735334 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-73658558-850e-45c2-9619-c95a05dab270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '589db10e33d7484eaa249fb83f4365b7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.742577 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-73658558-850e-45c2-9619-c95a05dab270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '589db10e33d7484eaa249fb83f4365b7'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:10.756584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1429/2793] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:04:10 2022] DELETE /identity/v3/users/589db10e33d7484eaa249fb83f4365b7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:04:11.024053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:11.025881 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:11.039443 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861ff7c-6683-47d5-9074-3481740969b1 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:11.097186 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228df240-e997-47ca-a14b-4dd2d69e6454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:11.098046 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228df240-e997-47ca-a14b-4dd2d69e6454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.098336 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228df240-e997-47ca-a14b-4dd2d69e6454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.098623 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228df240-e997-47ca-a14b-4dd2d69e6454 admin admin] PATH_INFO: `/v3/projects/dbf4caeeac114aa4b4a13ea07ddb2a1d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.099828 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228df240-e997-47ca-a14b-4dd2d69e6454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf4caeeac114aa4b4a13ea07ddb2a1d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:11.101834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228df240-e997-47ca-a14b-4dd2d69e6454 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:11.132440 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:11.149962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-228df240-e997-47ca-a14b-4dd2d69e6454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf4caeeac114aa4b4a13ea07ddb2a1d', 'request_id': 'req-228df240-e997-47ca-a14b-4dd2d69e6454'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:11.157330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-228df240-e997-47ca-a14b-4dd2d69e6454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf4caeeac114aa4b4a13ea07ddb2a1d', 'request_id': 'req-228df240-e997-47ca-a14b-4dd2d69e6454'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:11.163801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-228df240-e997-47ca-a14b-4dd2d69e6454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf4caeeac114aa4b4a13ea07ddb2a1d', 'request_id': 'req-228df240-e997-47ca-a14b-4dd2d69e6454'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:11.174651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-228df240-e997-47ca-a14b-4dd2d69e6454 admin admin] The token cache is being invalidate because project dbf4caeeac114aa4b4a13ea07ddb2a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:04:11.174944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-228df240-e997-47ca-a14b-4dd2d69e6454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:11.175603 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-228df240-e997-47ca-a14b-4dd2d69e6454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:11.177450 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1365/2794] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:04:11 2022] DELETE /identity/v3/projects/dbf4caeeac114aa4b4a13ea07ddb2a1d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:11.182429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74efd40-f3dd-4c3c-85c7-d3513d7d2342 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.182909 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74efd40-f3dd-4c3c-85c7-d3513d7d2342 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.183015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74efd40-f3dd-4c3c-85c7-d3513d7d2342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.201394 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-f74efd40-f3dd-4c3c-85c7-d3513d7d2342 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:11.202605 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f74efd40-f3dd-4c3c-85c7-d3513d7d2342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:11.261194 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1430/2795] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:11.270007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448dfaa-b23a-4c8d-8a89-7ca14c154fdb None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.270302 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448dfaa-b23a-4c8d-8a89-7ca14c154fdb None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.270586 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448dfaa-b23a-4c8d-8a89-7ca14c154fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.289611 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-2448dfaa-b23a-4c8d-8a89-7ca14c154fdb None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:11.290825 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-2448dfaa-b23a-4c8d-8a89-7ca14c154fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:11.299142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1366/2796] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:11.303549 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74efd40-f3dd-4c3c-85c7-d3513d7d2342 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:11.313229 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74efd40-f3dd-4c3c-85c7-d3513d7d2342 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:11.321330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03f29c-e6f3-4e2b-8997-d23a8220ef7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:11.321530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03f29c-e6f3-4e2b-8997-d23a8220ef7c admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.321784 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03f29c-e6f3-4e2b-8997-d23a8220ef7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.322061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03f29c-e6f3-4e2b-8997-d23a8220ef7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.322788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03f29c-e6f3-4e2b-8997-d23a8220ef7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80243) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:04:11.323133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03f29c-e6f3-4e2b-8997-d23a8220ef7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:11.324761 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03f29c-e6f3-4e2b-8997-d23a8220ef7c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:11.336812 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1431/2797] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:04:11.341151 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448dfaa-b23a-4c8d-8a89-7ca14c154fdb None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:11.350524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448dfaa-b23a-4c8d-8a89-7ca14c154fdb None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:11.355377 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64067bcd-a0e9-42ef-9b91-fa38237feaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:11.355965 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64067bcd-a0e9-42ef-9b91-fa38237feaf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.356248 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64067bcd-a0e9-42ef-9b91-fa38237feaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.356525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64067bcd-a0e9-42ef-9b91-fa38237feaf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.357252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64067bcd-a0e9-42ef-9b91-fa38237feaf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:11.359170 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64067bcd-a0e9-42ef-9b91-fa38237feaf4 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:11.386064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1367/2798] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:04:11 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:11.390385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03f29c-e6f3-4e2b-8997-d23a8220ef7c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:11.400522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03f29c-e6f3-4e2b-8997-d23a8220ef7c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:11.427191 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:11.458977 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fc7dd-eb1a-40fa-a213-2a7d606d4037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:11.459641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fc7dd-eb1a-40fa-a213-2a7d606d4037 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.459925 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fc7dd-eb1a-40fa-a213-2a7d606d4037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.460203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52fc7dd-eb1a-40fa-a213-2a7d606d4037 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.460972 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fc7dd-eb1a-40fa-a213-2a7d606d4037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:11.462851 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52fc7dd-eb1a-40fa-a213-2a7d606d4037 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:11.482360 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1432/2799] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:04:11 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:11.487169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64067bcd-a0e9-42ef-9b91-fa38237feaf4 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:11.497443 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64067bcd-a0e9-42ef-9b91-fa38237feaf4 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:11.502104 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73db23-5af0-4d79-a0df-bb7db5e2419e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:11.502755 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73db23-5af0-4d79-a0df-bb7db5e2419e admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.503049 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73db23-5af0-4d79-a0df-bb7db5e2419e admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.503330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73db23-5af0-4d79-a0df-bb7db5e2419e admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.504061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73db23-5af0-4d79-a0df-bb7db5e2419e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:11.505757 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73db23-5af0-4d79-a0df-bb7db5e2419e admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:11.516041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1368/2800] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:04:11.520571 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fc7dd-eb1a-40fa-a213-2a7d606d4037 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:11.531293 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52fc7dd-eb1a-40fa-a213-2a7d606d4037 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:11.536471 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5df111-cd46-4291-ad27-8ef744b02bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:11.537163 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5df111-cd46-4291-ad27-8ef744b02bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.537454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5df111-cd46-4291-ad27-8ef744b02bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.537738 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5df111-cd46-4291-ad27-8ef744b02bc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.538465 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5df111-cd46-4291-ad27-8ef744b02bc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:11.540050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5df111-cd46-4291-ad27-8ef744b02bc0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:11.550548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1433/2801] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:04:11.555019 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73db23-5af0-4d79-a0df-bb7db5e2419e admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:11.564941 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73db23-5af0-4d79-a0df-bb7db5e2419e admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:11.570079 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed1fbf-db97-46b2-8c0c-e67106907749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:11.570961 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed1fbf-db97-46b2-8c0c-e67106907749 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.571244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed1fbf-db97-46b2-8c0c-e67106907749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.571524 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ed1fbf-db97-46b2-8c0c-e67106907749 admin admin] PATH_INFO: `/v3/projects/37a98c2d2a9047089e7fa35a0afa39a1/users/c25635fb25c642548c692e41ef6dd733/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.593359 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed1fbf-db97-46b2-8c0c-e67106907749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37a98c2d2a9047089e7fa35a0afa39a1, user_id=c25635fb25c642548c692e41ef6dd733, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:11.596597 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ed1fbf-db97-46b2-8c0c-e67106907749 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:11.609338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1369/2802] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:04:11 2022] PUT /identity/v3/projects/37a98c2d2a9047089e7fa35a0afa39a1/users/c25635fb25c642548c692e41ef6dd733/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:11.619950 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d1ff6-a308-43e5-a2d5-f82102d9177f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.620249 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d1ff6-a308-43e5-a2d5-f82102d9177f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.620528 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d1ff6-a308-43e5-a2d5-f82102d9177f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.646032 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-333d1ff6-a308-43e5-a2d5-f82102d9177f None None] MFA Rules not processed for user `c25635fb25c642548c692e41ef6dd733`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:11.647248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-333d1ff6-a308-43e5-a2d5-f82102d9177f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:11.713170 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1434/2803] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:11.717696 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed1fbf-db97-46b2-8c0c-e67106907749 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:11.781772 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ed1fbf-db97-46b2-8c0c-e67106907749 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:11.786944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc5a0a-f880-4ab6-bdcc-27ea0d5f6f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:11.787608 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc5a0a-f880-4ab6-bdcc-27ea0d5f6f9c admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.787894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc5a0a-f880-4ab6-bdcc-27ea0d5f6f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.788175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc5a0a-f880-4ab6-bdcc-27ea0d5f6f9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.788906 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc5a0a-f880-4ab6-bdcc-27ea0d5f6f9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:11.790812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc5a0a-f880-4ab6-bdcc-27ea0d5f6f9c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:11.819748 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1370/2804] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:04:11 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:11.824216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d1ff6-a308-43e5-a2d5-f82102d9177f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:11.835789 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d1ff6-a308-43e5-a2d5-f82102d9177f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:11.895050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed8916-0b30-43ff-a5cc-d425d22c47c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:11.895639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed8916-0b30-43ff-a5cc-d425d22c47c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.896047 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed8916-0b30-43ff-a5cc-d425d22c47c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.896457 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed8916-0b30-43ff-a5cc-d425d22c47c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.897246 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed8916-0b30-43ff-a5cc-d425d22c47c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:11.899241 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed8916-0b30-43ff-a5cc-d425d22c47c8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:11.918811 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1435/2805] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:04:11 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:11.922992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc5a0a-f880-4ab6-bdcc-27ea0d5f6f9c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:11.933630 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc5a0a-f880-4ab6-bdcc-27ea0d5f6f9c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:11.938305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c66c1-1460-49b5-bef2-99b934f1dc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:11.938952 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c66c1-1460-49b5-bef2-99b934f1dc9c admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.939242 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c66c1-1460-49b5-bef2-99b934f1dc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.939525 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c66c1-1460-49b5-bef2-99b934f1dc9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.940278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c66c1-1460-49b5-bef2-99b934f1dc9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:11.941993 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c66c1-1460-49b5-bef2-99b934f1dc9c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:11.953033 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1371/2806] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:04:11.957458 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed8916-0b30-43ff-a5cc-d425d22c47c8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:11.967826 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed8916-0b30-43ff-a5cc-d425d22c47c8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:11.973157 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dd2c98-742e-47b7-88c6-560d94e0cef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:11.973813 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd2c98-742e-47b7-88c6-560d94e0cef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:11.974105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd2c98-742e-47b7-88c6-560d94e0cef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:11.974386 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dd2c98-742e-47b7-88c6-560d94e0cef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:11.975113 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd2c98-742e-47b7-88c6-560d94e0cef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:11.976731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dd2c98-742e-47b7-88c6-560d94e0cef0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:11.987615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1436/2807] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:04:11.991801 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c66c1-1460-49b5-bef2-99b934f1dc9c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:12.002076 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c66c1-1460-49b5-bef2-99b934f1dc9c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:12.011639 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf1456-ba5b-4ad0-9023-50c288147cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:12.012625 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf1456-ba5b-4ad0-9023-50c288147cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:12.012988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf1456-ba5b-4ad0-9023-50c288147cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:12.013265 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf1456-ba5b-4ad0-9023-50c288147cc5 admin admin] PATH_INFO: `/v3/projects/695bda784a8c49f7a28263ecff2887cc/users/a76421690b3a4f5cadf193c393b3ea8f/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:12.029185 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf1456-ba5b-4ad0-9023-50c288147cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=695bda784a8c49f7a28263ecff2887cc, user_id=a76421690b3a4f5cadf193c393b3ea8f, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:12.031826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf1456-ba5b-4ad0-9023-50c288147cc5 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:12.041398 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:12.046207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1372/2808] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:04:11 2022] PUT /identity/v3/projects/695bda784a8c49f7a28263ecff2887cc/users/a76421690b3a4f5cadf193c393b3ea8f/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:12.057393 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc594234-604a-4c9b-aef4-c981c1f84205 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:12.057686 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc594234-604a-4c9b-aef4-c981c1f84205 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:12.057990 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc594234-604a-4c9b-aef4-c981c1f84205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:12.084198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-cc594234-604a-4c9b-aef4-c981c1f84205 None None] MFA Rules not processed for user `a76421690b3a4f5cadf193c393b3ea8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:12.085508 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-cc594234-604a-4c9b-aef4-c981c1f84205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:12.089640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:12.151788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1437/2809] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:12.156275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf1456-ba5b-4ad0-9023-50c288147cc5 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:12.227640 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf1456-ba5b-4ad0-9023-50c288147cc5 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:12.232887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad614b64-078e-4f57-86cc-1b0d83965d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:12.233660 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad614b64-078e-4f57-86cc-1b0d83965d39 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:12.233944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad614b64-078e-4f57-86cc-1b0d83965d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:12.234219 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad614b64-078e-4f57-86cc-1b0d83965d39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:12.234905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad614b64-078e-4f57-86cc-1b0d83965d39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:12.236876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad614b64-078e-4f57-86cc-1b0d83965d39 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:12.267819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1373/2810] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:04:12 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:12.269958 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc594234-604a-4c9b-aef4-c981c1f84205 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:12.280607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc594234-604a-4c9b-aef4-c981c1f84205 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:12.349315 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7598e-b15d-4791-b025-987de1623ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:12.349988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7598e-b15d-4791-b025-987de1623ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:12.350268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7598e-b15d-4791-b025-987de1623ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:12.350697 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7598e-b15d-4791-b025-987de1623ea8 admin admin] PATH_INFO: `/v3/users` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:12.352818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7598e-b15d-4791-b025-987de1623ea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:12.353354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7598e-b15d-4791-b025-987de1623ea8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:12.374441 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1438/2811] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:04:12 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:12.379599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad614b64-078e-4f57-86cc-1b0d83965d39 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:12.390131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad614b64-078e-4f57-86cc-1b0d83965d39 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:12.395468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16400adc-f92e-4ff3-a957-4a7b19ca2869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:12.396127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16400adc-f92e-4ff3-a957-4a7b19ca2869 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:12.396412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16400adc-f92e-4ff3-a957-4a7b19ca2869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:12.396786 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16400adc-f92e-4ff3-a957-4a7b19ca2869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:12.397450 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16400adc-f92e-4ff3-a957-4a7b19ca2869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:12.399179 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16400adc-f92e-4ff3-a957-4a7b19ca2869 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:12.410523 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1374/2812] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:04:12.414749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7598e-b15d-4791-b025-987de1623ea8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:12.427635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7598e-b15d-4791-b025-987de1623ea8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:12.435688 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ffe25-7515-4988-a9d8-5df7dc57ede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:12.436583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ffe25-7515-4988-a9d8-5df7dc57ede4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:12.436891 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ffe25-7515-4988-a9d8-5df7dc57ede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:12.437180 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ffe25-7515-4988-a9d8-5df7dc57ede4 admin admin] PATH_INFO: `/v3/projects/422d65404ea14bc68e5677583b626dc9/users/4a2b85193eee46da8f1906871784a21e/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:12.456072 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ffe25-7515-4988-a9d8-5df7dc57ede4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=422d65404ea14bc68e5677583b626dc9, user_id=4a2b85193eee46da8f1906871784a21e, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:12.458780 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ffe25-7515-4988-a9d8-5df7dc57ede4 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:12.472121 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1439/2813] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:04:12 2022] PUT /identity/v3/projects/422d65404ea14bc68e5677583b626dc9/users/4a2b85193eee46da8f1906871784a21e/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:12.476196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16400adc-f92e-4ff3-a957-4a7b19ca2869 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:12.486542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16400adc-f92e-4ff3-a957-4a7b19ca2869 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:12.545341 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba231f97-819c-4a3b-aad9-b7a62dee508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:12.546000 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba231f97-819c-4a3b-aad9-b7a62dee508d admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:12.546284 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba231f97-819c-4a3b-aad9-b7a62dee508d admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:12.546564 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba231f97-819c-4a3b-aad9-b7a62dee508d admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:12.547285 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba231f97-819c-4a3b-aad9-b7a62dee508d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:12.549546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba231f97-819c-4a3b-aad9-b7a62dee508d admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:12.560475 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1375/2814] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:04:12.565488 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ffe25-7515-4988-a9d8-5df7dc57ede4 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:12.576125 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ffe25-7515-4988-a9d8-5df7dc57ede4 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:12.581248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d112a0-da78-4ff3-b40e-582ff828cfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:12.582135 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d112a0-da78-4ff3-b40e-582ff828cfa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:12.582422 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d112a0-da78-4ff3-b40e-582ff828cfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:12.582696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d112a0-da78-4ff3-b40e-582ff828cfa5 admin admin] PATH_INFO: `/v3/projects/422d65404ea14bc68e5677583b626dc9/users/4a2b85193eee46da8f1906871784a21e/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:12.584708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d112a0-da78-4ff3-b40e-582ff828cfa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=422d65404ea14bc68e5677583b626dc9, user_id=4a2b85193eee46da8f1906871784a21e, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:12.587515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d112a0-da78-4ff3-b40e-582ff828cfa5 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:12.598285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1440/2815] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:04:12 2022] PUT /identity/v3/projects/422d65404ea14bc68e5677583b626dc9/users/4a2b85193eee46da8f1906871784a21e/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:12.609544 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3b59f-ddf3-4f62-9933-3310e5e51c7f None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:12.609879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3b59f-ddf3-4f62-9933-3310e5e51c7f None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:12.610160 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3b59f-ddf3-4f62-9933-3310e5e51c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:12.643246 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-e9c3b59f-ddf3-4f62-9933-3310e5e51c7f None None] MFA Rules not processed for user `4a2b85193eee46da8f1906871784a21e`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:12.644730 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e9c3b59f-ddf3-4f62-9933-3310e5e51c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:12.719819 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:12.735662 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1376/2816] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:12.748020 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d112a0-da78-4ff3-b40e-582ff828cfa5 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:12.750081 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c4d112a0-da78-4ff3-b40e-582ff828cfa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:12.820751 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d112a0-da78-4ff3-b40e-582ff828cfa5 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:12.824461 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf05527-77e1-4115-a1f6-f444ebb60c1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:12.825142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf05527-77e1-4115-a1f6-f444ebb60c1d service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:12.825431 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf05527-77e1-4115-a1f6-f444ebb60c1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:12.825715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf05527-77e1-4115-a1f6-f444ebb60c1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:12.843385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf05527-77e1-4115-a1f6-f444ebb60c1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:12.845108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf05527-77e1-4115-a1f6-f444ebb60c1d service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:12.850806 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:12.858242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1441/2817] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:04:13.356269 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3b59f-ddf3-4f62-9933-3310e5e51c7f None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:13.357707 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-e9c3b59f-ddf3-4f62-9933-3310e5e51c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:13.370280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3b59f-ddf3-4f62-9933-3310e5e51c7f None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:13.373875 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1afc0b-7dfb-4c49-9714-22111014811d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:13.374507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1afc0b-7dfb-4c49-9714-22111014811d service cinder] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:13.374793 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1afc0b-7dfb-4c49-9714-22111014811d service cinder] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:13.375072 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1afc0b-7dfb-4c49-9714-22111014811d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:13.430740 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1afc0b-7dfb-4c49-9714-22111014811d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:13.432166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1afc0b-7dfb-4c49-9714-22111014811d service cinder] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:13.440943 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1377/2818] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:04:17.405008 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf05527-77e1-4115-a1f6-f444ebb60c1d service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:17.414309 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf05527-77e1-4115-a1f6-f444ebb60c1d service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:17.417980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eccd6e-1259-406a-a4e3-87b23dd95110 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:17.418613 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eccd6e-1259-406a-a4e3-87b23dd95110 service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:17.418902 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eccd6e-1259-406a-a4e3-87b23dd95110 service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:17.419185 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eccd6e-1259-406a-a4e3-87b23dd95110 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:17.476235 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eccd6e-1259-406a-a4e3-87b23dd95110 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:17.477709 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81eccd6e-1259-406a-a4e3-87b23dd95110 service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:17.487314 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1442/2819] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:04:19.693280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1afc0b-7dfb-4c49-9714-22111014811d service cinder] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:19.755927 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1afc0b-7dfb-4c49-9714-22111014811d service cinder] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:19.760453 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:19.761416 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:19.761709 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:19.761985 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] PATH_INFO: `/v3/users/c25635fb25c642548c692e41ef6dd733` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:19.763180 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c25635fb25c642548c692e41ef6dd733)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:19.765130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:19.812569 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c25635fb25c642548c692e41ef6dd733'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:19.813315 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c25635fb25c642548c692e41ef6dd733'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:19.813907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c25635fb25c642548c692e41ef6dd733'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:19.826346 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c25635fb25c642548c692e41ef6dd733'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:19.833661 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c25635fb25c642548c692e41ef6dd733'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:19.846176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1378/2820] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:04:19 2022] DELETE /identity/v3/users/c25635fb25c642548c692e41ef6dd733 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:19.850725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6af4d-eb36-4ee6-9f7a-534f703e947e None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:19.851201 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6af4d-eb36-4ee6-9f7a-534f703e947e None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:19.851336 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6af4d-eb36-4ee6-9f7a-534f703e947e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:19.868873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-edc6af4d-eb36-4ee6-9f7a-534f703e947e None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:19.870202 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-edc6af4d-eb36-4ee6-9f7a-534f703e947e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:19.938792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1443/2821] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:19.948887 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:19.950293 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:19.964935 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:20.008782 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1083eb-e245-4616-a4e3-917fd0189d6c admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:20.012339 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903eccd9-aa56-4259-bcfb-69b72d452c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:20.013069 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eccd9-aa56-4259-bcfb-69b72d452c44 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:20.013352 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eccd9-aa56-4259-bcfb-69b72d452c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:20.013627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eccd9-aa56-4259-bcfb-69b72d452c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:20.023649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903eccd9-aa56-4259-bcfb-69b72d452c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:20.025127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903eccd9-aa56-4259-bcfb-69b72d452c44 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:20.034536 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1379/2822] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:04:20.305425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6af4d-eb36-4ee6-9f7a-534f703e947e None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:20.306917 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-edc6af4d-eb36-4ee6-9f7a-534f703e947e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:20.321316 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6af4d-eb36-4ee6-9f7a-534f703e947e None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:20.326346 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:20.327218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:20.327505 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:20.327784 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] PATH_INFO: `/v3/projects/37a98c2d2a9047089e7fa35a0afa39a1` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:20.329036 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=37a98c2d2a9047089e7fa35a0afa39a1)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:20.331639 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:20.378410 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37a98c2d2a9047089e7fa35a0afa39a1', 'request_id': 'req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:20.385268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37a98c2d2a9047089e7fa35a0afa39a1', 'request_id': 'req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:20.390700 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37a98c2d2a9047089e7fa35a0afa39a1', 'request_id': 'req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:20.402060 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] The token cache is being invalidate because project 37a98c2d2a9047089e7fa35a0afa39a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:04:20.402343 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:20.402995 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:20.404515 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1444/2823] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:04:20 2022] DELETE /identity/v3/projects/37a98c2d2a9047089e7fa35a0afa39a1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:20.408770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903eccd9-aa56-4259-bcfb-69b72d452c44 service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:20.419137 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903eccd9-aa56-4259-bcfb-69b72d452c44 service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:20.475098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:20.475979 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:20.476311 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:20.476627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] PATH_INFO: `/v3/users/a76421690b3a4f5cadf193c393b3ea8f` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:20.478100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76421690b3a4f5cadf193c393b3ea8f)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:20.480056 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:20.527831 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76421690b3a4f5cadf193c393b3ea8f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:20.528487 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76421690b3a4f5cadf193c393b3ea8f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:20.529133 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76421690b3a4f5cadf193c393b3ea8f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:20.540882 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76421690b3a4f5cadf193c393b3ea8f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:20.547598 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76421690b3a4f5cadf193c393b3ea8f'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:20.560381 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1380/2824] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:04:20 2022] DELETE /identity/v3/users/a76421690b3a4f5cadf193c393b3ea8f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:20.878738 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:20.880387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:20.894237 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f1bdfd-703e-4a52-bf00-78b729c6abfb admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:20.916288 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:20.960385 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:20.961278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:20.961567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:20.961846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] PATH_INFO: `/v3/projects/695bda784a8c49f7a28263ecff2887cc` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:20.963061 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=695bda784a8c49f7a28263ecff2887cc)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:20.965141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:21.014974 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '695bda784a8c49f7a28263ecff2887cc', 'request_id': 'req-59f219fe-7368-43b0-9911-9d452adc3486'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.024454 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '695bda784a8c49f7a28263ecff2887cc', 'request_id': 'req-59f219fe-7368-43b0-9911-9d452adc3486'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.032127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '695bda784a8c49f7a28263ecff2887cc', 'request_id': 'req-59f219fe-7368-43b0-9911-9d452adc3486'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.044756 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] The token cache is being invalidate because project 695bda784a8c49f7a28263ecff2887cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:04:21.045049 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.045710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.047244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1445/2825] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:04:20 2022] DELETE /identity/v3/projects/695bda784a8c49f7a28263ecff2887cc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:21.051627 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:21.053245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:21.066289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a307df-bd01-447d-898b-0dd8dc89c6bc admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:21.102047 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:21.130513 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14844d61-fb45-4d43-85ff-8863537c8db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:21.131390 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14844d61-fb45-4d43-85ff-8863537c8db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:21.131675 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14844d61-fb45-4d43-85ff-8863537c8db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:21.131957 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14844d61-fb45-4d43-85ff-8863537c8db2 admin admin] PATH_INFO: `/v3/users/4a2b85193eee46da8f1906871784a21e` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:21.133202 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14844d61-fb45-4d43-85ff-8863537c8db2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a2b85193eee46da8f1906871784a21e)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:21.135118 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14844d61-fb45-4d43-85ff-8863537c8db2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:21.187654 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-14844d61-fb45-4d43-85ff-8863537c8db2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a2b85193eee46da8f1906871784a21e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.188244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-14844d61-fb45-4d43-85ff-8863537c8db2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a2b85193eee46da8f1906871784a21e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.188911 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-14844d61-fb45-4d43-85ff-8863537c8db2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a2b85193eee46da8f1906871784a21e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.202693 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-14844d61-fb45-4d43-85ff-8863537c8db2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a2b85193eee46da8f1906871784a21e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.210670 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-14844d61-fb45-4d43-85ff-8863537c8db2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a2b85193eee46da8f1906871784a21e'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.224122 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1381/2826] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:04:21 2022] DELETE /identity/v3/users/4a2b85193eee46da8f1906871784a21e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:21.530338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:21.531985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:21.550584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f219fe-7368-43b0-9911-9d452adc3486 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:21.610429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6dd397-f676-4660-8b5e-bf8d21d70139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:21.611300 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dd397-f676-4660-8b5e-bf8d21d70139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:21.611594 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dd397-f676-4660-8b5e-bf8d21d70139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:21.612584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6dd397-f676-4660-8b5e-bf8d21d70139 admin admin] PATH_INFO: `/v3/projects/422d65404ea14bc68e5677583b626dc9` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:21.613956 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6dd397-f676-4660-8b5e-bf8d21d70139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=422d65404ea14bc68e5677583b626dc9)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:21.615864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6dd397-f676-4660-8b5e-bf8d21d70139 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:21.665567 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2a6dd397-f676-4660-8b5e-bf8d21d70139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '422d65404ea14bc68e5677583b626dc9', 'request_id': 'req-2a6dd397-f676-4660-8b5e-bf8d21d70139'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.669970 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:21.674188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2a6dd397-f676-4660-8b5e-bf8d21d70139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '422d65404ea14bc68e5677583b626dc9', 'request_id': 'req-2a6dd397-f676-4660-8b5e-bf8d21d70139'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.686955 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2a6dd397-f676-4660-8b5e-bf8d21d70139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '422d65404ea14bc68e5677583b626dc9', 'request_id': 'req-2a6dd397-f676-4660-8b5e-bf8d21d70139'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.699967 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2a6dd397-f676-4660-8b5e-bf8d21d70139 admin admin] The token cache is being invalidate because project 422d65404ea14bc68e5677583b626dc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:04:21.700198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2a6dd397-f676-4660-8b5e-bf8d21d70139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.700941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-2a6dd397-f676-4660-8b5e-bf8d21d70139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:21.702512 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1446/2827] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:04:21 2022] DELETE /identity/v3/projects/422d65404ea14bc68e5677583b626dc9 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:21.708482 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10064a-851d-4854-9664-bd760329a4f6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:21.708876 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10064a-851d-4854-9664-bd760329a4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:21.709231 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10064a-851d-4854-9664-bd760329a4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:21.727469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1e10064a-851d-4854-9664-bd760329a4f6 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:21.728879 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1e10064a-851d-4854-9664-bd760329a4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:21.795708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1382/2828] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:04:21.804280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35715066-9169-4611-bb26-55be12a1f978 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:21.804574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35715066-9169-4611-bb26-55be12a1f978 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:21.804903 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35715066-9169-4611-bb26-55be12a1f978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:21.823781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-35715066-9169-4611-bb26-55be12a1f978 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:21.825054 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-35715066-9169-4611-bb26-55be12a1f978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:21.834096 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1447/2829] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:21.841844 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10064a-851d-4854-9664-bd760329a4f6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:21.853971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10064a-851d-4854-9664-bd760329a4f6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:21.859775 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4b724-72d2-4841-885d-fca1b3c49621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:21.860469 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4b724-72d2-4841-885d-fca1b3c49621 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:21.860834 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4b724-72d2-4841-885d-fca1b3c49621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:21.861147 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4b724-72d2-4841-885d-fca1b3c49621 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:21.861907 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4b724-72d2-4841-885d-fca1b3c49621 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:04:21.862207 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4b724-72d2-4841-885d-fca1b3c49621 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:21.863841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4b724-72d2-4841-885d-fca1b3c49621 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:21.882942 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1383/2830] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:04:21.887262 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35715066-9169-4611-bb26-55be12a1f978 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:21.899156 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35715066-9169-4611-bb26-55be12a1f978 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:21.907681 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48029439-4188-4108-ae1d-cd907b957a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:21.908338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48029439-4188-4108-ae1d-cd907b957a14 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:21.908643 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48029439-4188-4108-ae1d-cd907b957a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:21.909548 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48029439-4188-4108-ae1d-cd907b957a14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:21.910253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48029439-4188-4108-ae1d-cd907b957a14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:21.912138 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48029439-4188-4108-ae1d-cd907b957a14 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:21.943574 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1448/2831] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:04:21 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:21.948127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4b724-72d2-4841-885d-fca1b3c49621 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:21.961968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4b724-72d2-4841-885d-fca1b3c49621 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:22.026254 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:22.057885 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7dfc7-c27f-49d2-945e-6a80b4b228ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:22.062644 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7dfc7-c27f-49d2-945e-6a80b4b228ab admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:22.064444 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7dfc7-c27f-49d2-945e-6a80b4b228ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:22.065497 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df7dfc7-c27f-49d2-945e-6a80b4b228ab admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:22.066406 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7dfc7-c27f-49d2-945e-6a80b4b228ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:22.069420 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df7dfc7-c27f-49d2-945e-6a80b4b228ab admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:22.093127 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1384/2832] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:04:21 2022] POST /identity/v3/users => generated 291 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:22.097807 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48029439-4188-4108-ae1d-cd907b957a14 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:22.108565 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48029439-4188-4108-ae1d-cd907b957a14 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:22.114039 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f34b20-ccad-43aa-ad4f-ff921b8a63a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:22.114708 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f34b20-ccad-43aa-ad4f-ff921b8a63a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:22.114992 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f34b20-ccad-43aa-ad4f-ff921b8a63a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:22.115269 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f34b20-ccad-43aa-ad4f-ff921b8a63a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:22.116000 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f34b20-ccad-43aa-ad4f-ff921b8a63a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:22.117621 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f34b20-ccad-43aa-ad4f-ff921b8a63a0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:22.129601 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1449/2833] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:04:22.133515 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7dfc7-c27f-49d2-945e-6a80b4b228ab admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:22.144815 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df7dfc7-c27f-49d2-945e-6a80b4b228ab admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:22.151175 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d57b127-bdf0-429e-8daf-be311c42e002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:22.151948 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57b127-bdf0-429e-8daf-be311c42e002 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:22.152305 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57b127-bdf0-429e-8daf-be311c42e002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:22.152653 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d57b127-bdf0-429e-8daf-be311c42e002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:22.153480 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57b127-bdf0-429e-8daf-be311c42e002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:22.155165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d57b127-bdf0-429e-8daf-be311c42e002 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:22.170702 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1385/2834] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:04:22.172347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f34b20-ccad-43aa-ad4f-ff921b8a63a0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:22.185720 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f34b20-ccad-43aa-ad4f-ff921b8a63a0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:22.192195 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4a3a-dedf-461c-b839-968e4c789a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:22.193846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4a3a-dedf-461c-b839-968e4c789a3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:22.193846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4a3a-dedf-461c-b839-968e4c789a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:22.193846 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e4a3a-dedf-461c-b839-968e4c789a3e admin admin] PATH_INFO: `/v3/projects/1dec574a91fb4357b2454f4d17fb4036/users/034ddfc7ec6e412ca2bbb931cf4758e9/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:22.211320 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4a3a-dedf-461c-b839-968e4c789a3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dec574a91fb4357b2454f4d17fb4036, user_id=034ddfc7ec6e412ca2bbb931cf4758e9, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:22.214142 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e4a3a-dedf-461c-b839-968e4c789a3e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:22.238853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1450/2835] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:04:22 2022] PUT /identity/v3/projects/1dec574a91fb4357b2454f4d17fb4036/users/034ddfc7ec6e412ca2bbb931cf4758e9/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:22.243322 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e757c3e-4bcf-49e1-a996-038feceabce5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:22.243680 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e757c3e-4bcf-49e1-a996-038feceabce5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:22.243966 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e757c3e-4bcf-49e1-a996-038feceabce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:22.272448 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-9e757c3e-4bcf-49e1-a996-038feceabce5 None None] MFA Rules not processed for user `034ddfc7ec6e412ca2bbb931cf4758e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:22.273968 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9e757c3e-4bcf-49e1-a996-038feceabce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:22.345638 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1386/2836] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:22.357607 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4a3a-dedf-461c-b839-968e4c789a3e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:22.359321 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f13e4a3a-dedf-461c-b839-968e4c789a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:22.417348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e4a3a-dedf-461c-b839-968e4c789a3e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:22.421108 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9169f-2724-41a0-8025-99373027bc5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:22.422188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9169f-2724-41a0-8025-99373027bc5f service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:22.422188 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9169f-2724-41a0-8025-99373027bc5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:22.422480 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9169f-2724-41a0-8025-99373027bc5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:22.431221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9169f-2724-41a0-8025-99373027bc5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:22.432647 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e9169f-2724-41a0-8025-99373027bc5f service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:22.441696 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1451/2837] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:04:22.461061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e757c3e-4bcf-49e1-a996-038feceabce5 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:22.530301 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e757c3e-4bcf-49e1-a996-038feceabce5 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:22.535635 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:22.536508 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:22.536829 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:22.537115 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] PATH_INFO: `/v3/users/034ddfc7ec6e412ca2bbb931cf4758e9` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:22.538338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=034ddfc7ec6e412ca2bbb931cf4758e9)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:22.540258 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:22.594612 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '034ddfc7ec6e412ca2bbb931cf4758e9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:22.595292 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '034ddfc7ec6e412ca2bbb931cf4758e9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:22.595913 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '034ddfc7ec6e412ca2bbb931cf4758e9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:22.601109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:22.617343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '034ddfc7ec6e412ca2bbb931cf4758e9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:22.626460 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '034ddfc7ec6e412ca2bbb931cf4758e9'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:22.639891 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1387/2838] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:04:22 2022] DELETE /identity/v3/users/034ddfc7ec6e412ca2bbb931cf4758e9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:22.645509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64797d8d-1093-438a-aa73-60c24bde2dbf None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:22.646023 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64797d8d-1093-438a-aa73-60c24bde2dbf None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:22.646506 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64797d8d-1093-438a-aa73-60c24bde2dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:22.673190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-64797d8d-1093-438a-aa73-60c24bde2dbf None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:22.674562 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-64797d8d-1093-438a-aa73-60c24bde2dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:22.735347 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1452/2839] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:22.745255 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:22.746715 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:22.808493 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629d591-14c5-429f-9ad9-288412c05d57 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:22.812288 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb46cd-da5a-4d26-8bb6-d51061d2b254 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:22.812980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb46cd-da5a-4d26-8bb6-d51061d2b254 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:22.813272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb46cd-da5a-4d26-8bb6-d51061d2b254 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:22.813556 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb46cd-da5a-4d26-8bb6-d51061d2b254 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:22.823626 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb46cd-da5a-4d26-8bb6-d51061d2b254 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:22.825106 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb46cd-da5a-4d26-8bb6-d51061d2b254 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:22.839651 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1388/2840] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:04:23.103988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64797d8d-1093-438a-aa73-60c24bde2dbf None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:23.105635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-64797d8d-1093-438a-aa73-60c24bde2dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:23.118397 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64797d8d-1093-438a-aa73-60c24bde2dbf None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:23.123313 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:23.124165 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:23.124489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:23.124740 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a admin admin] PATH_INFO: `/v3/projects/1dec574a91fb4357b2454f4d17fb4036` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:23.125933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dec574a91fb4357b2454f4d17fb4036)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:23.128462 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:23.145191 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:23.178029 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dec574a91fb4357b2454f4d17fb4036', 'request_id': 'req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:23.186860 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dec574a91fb4357b2454f4d17fb4036', 'request_id': 'req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:23.193369 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dec574a91fb4357b2454f4d17fb4036', 'request_id': 'req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:23.204562 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a admin admin] The token cache is being invalidate because project 1dec574a91fb4357b2454f4d17fb4036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:04:23.204884 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:23.205652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0bbe6169-36f9-42cc-821d-7f73afbbcc5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:23.207190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1453/2841] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:04:23 2022] DELETE /identity/v3/projects/1dec574a91fb4357b2454f4d17fb4036 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:23.213592 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496ef73-af67-424f-a923-4a71709e08ba None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:23.214081 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496ef73-af67-424f-a923-4a71709e08ba None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:23.214200 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496ef73-af67-424f-a923-4a71709e08ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:23.231389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-1496ef73-af67-424f-a923-4a71709e08ba None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:23.232532 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-1496ef73-af67-424f-a923-4a71709e08ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:23.300517 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1389/2842] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:23.308579 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43593d86-26c5-4037-a72f-753c9cdbbe4f None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:23.308988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43593d86-26c5-4037-a72f-753c9cdbbe4f None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:23.309359 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43593d86-26c5-4037-a72f-753c9cdbbe4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:23.331490 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-43593d86-26c5-4037-a72f-753c9cdbbe4f None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:23.332876 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-43593d86-26c5-4037-a72f-753c9cdbbe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:23.342192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1454/2843] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:23.346280 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496ef73-af67-424f-a923-4a71709e08ba None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:23.356514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496ef73-af67-424f-a923-4a71709e08ba None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:23.361281 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c506af2-66ea-4d41-a615-f1a7545cf0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:23.362600 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c506af2-66ea-4d41-a615-f1a7545cf0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:23.363071 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c506af2-66ea-4d41-a615-f1a7545cf0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:23.363331 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c506af2-66ea-4d41-a615-f1a7545cf0f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:23.364131 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c506af2-66ea-4d41-a615-f1a7545cf0f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:04:23.364617 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c506af2-66ea-4d41-a615-f1a7545cf0f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:23.366222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c506af2-66ea-4d41-a615-f1a7545cf0f7 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:23.381561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1390/2844] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:04:23.383831 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43593d86-26c5-4037-a72f-753c9cdbbe4f None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:23.393372 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43593d86-26c5-4037-a72f-753c9cdbbe4f None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:23.398187 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8e2c5-b963-4831-90b9-18db870d3af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:23.398840 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8e2c5-b963-4831-90b9-18db870d3af0 admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:23.399127 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8e2c5-b963-4831-90b9-18db870d3af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:23.399410 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8e2c5-b963-4831-90b9-18db870d3af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:23.400107 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8e2c5-b963-4831-90b9-18db870d3af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:23.402040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a8e2c5-b963-4831-90b9-18db870d3af0 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:23.432252 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1455/2845] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:04:23 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:23.443112 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c506af2-66ea-4d41-a615-f1a7545cf0f7 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:23.454093 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c506af2-66ea-4d41-a615-f1a7545cf0f7 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:23.513992 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04b359-cee0-4815-bd06-672bf62a92fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:23.514656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04b359-cee0-4815-bd06-672bf62a92fd admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:23.515053 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04b359-cee0-4815-bd06-672bf62a92fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:23.515193 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae04b359-cee0-4815-bd06-672bf62a92fd admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:23.515892 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04b359-cee0-4815-bd06-672bf62a92fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:23.517843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae04b359-cee0-4815-bd06-672bf62a92fd admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:23.525726 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:23.541607 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1391/2846] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:04:23 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:23.545652 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8e2c5-b963-4831-90b9-18db870d3af0 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:23.557087 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a8e2c5-b963-4831-90b9-18db870d3af0 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:23.561725 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f67cce-34b2-4608-a05f-b92a0878703e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:23.562844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f67cce-34b2-4608-a05f-b92a0878703e admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:23.562844 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f67cce-34b2-4608-a05f-b92a0878703e admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:23.563090 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f67cce-34b2-4608-a05f-b92a0878703e admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:23.563655 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f67cce-34b2-4608-a05f-b92a0878703e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:23.565307 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f67cce-34b2-4608-a05f-b92a0878703e admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:23.576944 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1456/2847] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:04:23.581183 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04b359-cee0-4815-bd06-672bf62a92fd admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:23.591568 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae04b359-cee0-4815-bd06-672bf62a92fd admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:23.596866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4482cc-ae34-4192-973a-d892a161a79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:23.597527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4482cc-ae34-4192-973a-d892a161a79a admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:23.597812 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4482cc-ae34-4192-973a-d892a161a79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:23.598089 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4482cc-ae34-4192-973a-d892a161a79a admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:23.598817 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4482cc-ae34-4192-973a-d892a161a79a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:23.600443 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4482cc-ae34-4192-973a-d892a161a79a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:23.611152 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1392/2848] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:04:23.615945 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f67cce-34b2-4608-a05f-b92a0878703e admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:23.625873 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f67cce-34b2-4608-a05f-b92a0878703e admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:23.630485 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94932-e77e-4f0d-9dd0-1ceaca0d47ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:23.631350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94932-e77e-4f0d-9dd0-1ceaca0d47ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:23.632227 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94932-e77e-4f0d-9dd0-1ceaca0d47ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:23.632514 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd94932-e77e-4f0d-9dd0-1ceaca0d47ae admin admin] PATH_INFO: `/v3/projects/eee09804a46146b6934f0d738a7134d8/users/db5f0bfe5b6a403b86d1410a1e55f50d/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:23.647083 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd94932-e77e-4f0d-9dd0-1ceaca0d47ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=eee09804a46146b6934f0d738a7134d8, user_id=db5f0bfe5b6a403b86d1410a1e55f50d, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:23.649758 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd94932-e77e-4f0d-9dd0-1ceaca0d47ae admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:23.670272 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1457/2849] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:04:23 2022] PUT /identity/v3/projects/eee09804a46146b6934f0d738a7134d8/users/db5f0bfe5b6a403b86d1410a1e55f50d/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:23.681355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea78051-4cc8-47e2-80b9-3c545c3c818c None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:23.681647 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea78051-4cc8-47e2-80b9-3c545c3c818c None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:23.681926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea78051-4cc8-47e2-80b9-3c545c3c818c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:23.707655 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-8ea78051-4cc8-47e2-80b9-3c545c3c818c None None] MFA Rules not processed for user `db5f0bfe5b6a403b86d1410a1e55f50d`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:23.708894 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8ea78051-4cc8-47e2-80b9-3c545c3c818c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:23.772705 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1393/2850] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:23.784260 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94932-e77e-4f0d-9dd0-1ceaca0d47ae admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:23.786015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-8dd94932-e77e-4f0d-9dd0-1ceaca0d47ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:23.843616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd94932-e77e-4f0d-9dd0-1ceaca0d47ae admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:23.848584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074af68-9793-417c-953e-9020f43948de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:23.848766 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074af68-9793-417c-953e-9020f43948de service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:23.848988 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074af68-9793-417c-953e-9020f43948de service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:23.849270 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074af68-9793-417c-953e-9020f43948de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:23.858805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074af68-9793-417c-953e-9020f43948de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:23.860524 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074af68-9793-417c-953e-9020f43948de service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:23.870161 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1458/2851] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:04:23.895380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea78051-4cc8-47e2-80b9-3c545c3c818c None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:23.965273 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea78051-4cc8-47e2-80b9-3c545c3c818c None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:23.969883 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:23.970763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:23.971051 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:23.971898 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] PATH_INFO: `/v3/users/db5f0bfe5b6a403b86d1410a1e55f50d` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:23.973164 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db5f0bfe5b6a403b86d1410a1e55f50d)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:23.975091 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:24.024558 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db5f0bfe5b6a403b86d1410a1e55f50d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:24.025169 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db5f0bfe5b6a403b86d1410a1e55f50d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:24.025783 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db5f0bfe5b6a403b86d1410a1e55f50d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:24.038628 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db5f0bfe5b6a403b86d1410a1e55f50d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:24.043345 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:24.046027 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db5f0bfe5b6a403b86d1410a1e55f50d'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:24.063324 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1394/2852] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:04:23 2022] DELETE /identity/v3/users/db5f0bfe5b6a403b86d1410a1e55f50d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:24.067825 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03448df8-9115-4357-b8bf-3a356337cb9c None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:24.068122 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03448df8-9115-4357-b8bf-3a356337cb9c None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:24.068408 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03448df8-9115-4357-b8bf-3a356337cb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:24.088495 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-03448df8-9115-4357-b8bf-3a356337cb9c None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:24.089787 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-03448df8-9115-4357-b8bf-3a356337cb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:24.135221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:24.156128 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1459/2853] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:24.166007 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:24.168971 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:24.228530 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fbc5f-d70b-43da-89ea-878d1c19d424 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:24.232418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a9a21-1e15-41ec-81e8-a315fc7fea70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:24.232418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a9a21-1e15-41ec-81e8-a315fc7fea70 service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:24.232418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a9a21-1e15-41ec-81e8-a315fc7fea70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:24.232418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a9a21-1e15-41ec-81e8-a315fc7fea70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:24.244011 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a9a21-1e15-41ec-81e8-a315fc7fea70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:24.245496 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a9a21-1e15-41ec-81e8-a315fc7fea70 service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:24.257850 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1395/2854] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:04:24.582218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03448df8-9115-4357-b8bf-3a356337cb9c None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:24.583941 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-03448df8-9115-4357-b8bf-3a356337cb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:24.597164 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03448df8-9115-4357-b8bf-3a356337cb9c None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:24.603019 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486518db-d85d-47cb-b38e-55132ecdf2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:24.604015 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486518db-d85d-47cb-b38e-55132ecdf2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:24.604302 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486518db-d85d-47cb-b38e-55132ecdf2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:24.604583 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486518db-d85d-47cb-b38e-55132ecdf2c9 admin admin] PATH_INFO: `/v3/projects/eee09804a46146b6934f0d738a7134d8` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:24.605832 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486518db-d85d-47cb-b38e-55132ecdf2c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eee09804a46146b6934f0d738a7134d8)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:24.607979 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486518db-d85d-47cb-b38e-55132ecdf2c9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:24.670742 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-486518db-d85d-47cb-b38e-55132ecdf2c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eee09804a46146b6934f0d738a7134d8', 'request_id': 'req-486518db-d85d-47cb-b38e-55132ecdf2c9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:24.679504 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-486518db-d85d-47cb-b38e-55132ecdf2c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eee09804a46146b6934f0d738a7134d8', 'request_id': 'req-486518db-d85d-47cb-b38e-55132ecdf2c9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:24.687354 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-486518db-d85d-47cb-b38e-55132ecdf2c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eee09804a46146b6934f0d738a7134d8', 'request_id': 'req-486518db-d85d-47cb-b38e-55132ecdf2c9'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:24.709815 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-486518db-d85d-47cb-b38e-55132ecdf2c9 admin admin] The token cache is being invalidate because project eee09804a46146b6934f0d738a7134d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:04:24.711496 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-486518db-d85d-47cb-b38e-55132ecdf2c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:24.712278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-486518db-d85d-47cb-b38e-55132ecdf2c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:24.714244 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1460/2855] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:04:24 2022] DELETE /identity/v3/projects/eee09804a46146b6934f0d738a7134d8 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:24.719330 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b81981-29f7-474d-a820-13cb965295fa None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:24.720238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b81981-29f7-474d-a820-13cb965295fa None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:24.720238 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b81981-29f7-474d-a820-13cb965295fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:24.740403 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-55b81981-29f7-474d-a820-13cb965295fa None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:24.741745 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-55b81981-29f7-474d-a820-13cb965295fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:24.832527 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1396/2856] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:24.837923 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eb77e-9940-4caf-b915-a698a6913e35 None None] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:24.838214 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eb77e-9940-4caf-b915-a698a6913e35 None None] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:24.838493 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eb77e-9940-4caf-b915-a698a6913e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:24.858040 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.auth.core [None req-5c5eb77e-9940-4caf-b915-a698a6913e35 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80243) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:24.859233 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-5c5eb77e-9940-4caf-b915-a698a6913e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:24.868416 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1461/2857] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:24.872537 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b81981-29f7-474d-a820-13cb965295fa None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:24.883028 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b81981-29f7-474d-a820-13cb965295fa None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:24.888382 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e1ba5-da5b-45f7-ac31-d928011cd084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:24.889083 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e1ba5-da5b-45f7-ac31-d928011cd084 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:24.889369 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e1ba5-da5b-45f7-ac31-d928011cd084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:24.889649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e1ba5-da5b-45f7-ac31-d928011cd084 admin admin] PATH_INFO: `/v3/domains` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:24.890379 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e1ba5-da5b-45f7-ac31-d928011cd084 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80244) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 12:04:24.890691 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e1ba5-da5b-45f7-ac31-d928011cd084 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:24.892303 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e1ba5-da5b-45f7-ac31-d928011cd084 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:24.904656 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1397/2858] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 23 12:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 12:04:24.909799 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5eb77e-9940-4caf-b915-a698a6913e35 None None] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:24.919425 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5eb77e-9940-4caf-b915-a698a6913e35 None None] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:24.926953 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a43e8-fb69-467f-a11c-f7d9bd3c320a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:24.928130 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a43e8-fb69-467f-a11c-f7d9bd3c320a admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:24.928633 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a43e8-fb69-467f-a11c-f7d9bd3c320a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:24.929190 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a43e8-fb69-467f-a11c-f7d9bd3c320a admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:24.930376 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a43e8-fb69-467f-a11c-f7d9bd3c320a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:24.932733 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a43e8-fb69-467f-a11c-f7d9bd3c320a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:24.959198 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1462/2859] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:04:24 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 12:04:24.963545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e1ba5-da5b-45f7-ac31-d928011cd084 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:24.978431 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e1ba5-da5b-45f7-ac31-d928011cd084 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:25.042203 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06825054-bdcd-45fa-b074-8ee2e83452c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:25.042988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06825054-bdcd-45fa-b074-8ee2e83452c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:25.043270 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06825054-bdcd-45fa-b074-8ee2e83452c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:25.043545 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06825054-bdcd-45fa-b074-8ee2e83452c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:25.044258 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06825054-bdcd-45fa-b074-8ee2e83452c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:25.046338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06825054-bdcd-45fa-b074-8ee2e83452c2 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:25.072604 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1398/2860] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:04:24 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:25.074286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a43e8-fb69-467f-a11c-f7d9bd3c320a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:25.085896 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a43e8-fb69-467f-a11c-f7d9bd3c320a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:25.090475 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a80e19-f5ad-4a3b-b214-2e16bf99b719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:25.091139 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80e19-f5ad-4a3b-b214-2e16bf99b719 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:25.091442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80e19-f5ad-4a3b-b214-2e16bf99b719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:25.091798 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a80e19-f5ad-4a3b-b214-2e16bf99b719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:25.092601 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a80e19-f5ad-4a3b-b214-2e16bf99b719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:25.094242 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a80e19-f5ad-4a3b-b214-2e16bf99b719 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:25.106234 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1463/2861] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 12:04:25.110245 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06825054-bdcd-45fa-b074-8ee2e83452c2 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:25.121407 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06825054-bdcd-45fa-b074-8ee2e83452c2 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:25.126162 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f56298-3546-4146-9f9c-9a640f325469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:25.126823 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f56298-3546-4146-9f9c-9a640f325469 admin admin] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:25.127109 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f56298-3546-4146-9f9c-9a640f325469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:25.127416 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f56298-3546-4146-9f9c-9a640f325469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:25.128393 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f56298-3546-4146-9f9c-9a640f325469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:25.129905 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f56298-3546-4146-9f9c-9a640f325469 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:25.131334 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:25.141962 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1399/2862] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:04:25.146407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a80e19-f5ad-4a3b-b214-2e16bf99b719 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:25.157967 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a80e19-f5ad-4a3b-b214-2e16bf99b719 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:25.163629 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961bb47-dc71-4e76-aa09-07b14ffc9416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:25.164547 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961bb47-dc71-4e76-aa09-07b14ffc9416 admin admin] REQUEST_METHOD: `PUT` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:25.164874 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961bb47-dc71-4e76-aa09-07b14ffc9416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:25.165216 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961bb47-dc71-4e76-aa09-07b14ffc9416 admin admin] PATH_INFO: `/v3/projects/c41820d2b5eb4dedb313bcef88259676/users/8731c4e543814bcf80075cd315a7b7ff/roles/20703235b4c543bc9c51fb7c26da270d` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:25.182905 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961bb47-dc71-4e76-aa09-07b14ffc9416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c41820d2b5eb4dedb313bcef88259676, user_id=8731c4e543814bcf80075cd315a7b7ff, role_id=20703235b4c543bc9c51fb7c26da270d)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:25.185636 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961bb47-dc71-4e76-aa09-07b14ffc9416 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:25.200132 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1464/2863] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:04:25 2022] PUT /identity/v3/projects/c41820d2b5eb4dedb313bcef88259676/users/8731c4e543814bcf80075cd315a7b7ff/roles/20703235b4c543bc9c51fb7c26da270d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:04:25.205335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251ff01-19f5-45e9-8379-0118b270cd50 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:25.206312 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251ff01-19f5-45e9-8379-0118b270cd50 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:25.206492 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251ff01-19f5-45e9-8379-0118b270cd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:25.224546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-3251ff01-19f5-45e9-8379-0118b270cd50 None None] MFA Rules not processed for user `86b831b5471b48e6bf2bf9ce771279b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:25.225826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-3251ff01-19f5-45e9-8379-0118b270cd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:25.284980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1400/2864] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:25.296366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961bb47-dc71-4e76-aa09-07b14ffc9416 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:25.298144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-c961bb47-dc71-4e76-aa09-07b14ffc9416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:25.357594 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961bb47-dc71-4e76-aa09-07b14ffc9416 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:25.361154 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de64807-09c7-4ace-8e8c-9ca62ff1deb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:25.362368 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de64807-09c7-4ace-8e8c-9ca62ff1deb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:25.362659 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de64807-09c7-4ace-8e8c-9ca62ff1deb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:25.362966 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de64807-09c7-4ace-8e8c-9ca62ff1deb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:25.372387 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de64807-09c7-4ace-8e8c-9ca62ff1deb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:25.373864 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de64807-09c7-4ace-8e8c-9ca62ff1deb3 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:25.384279 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1465/2865] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:04:30.308090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee5e00-1832-4369-b436-56bbaef13e9a None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:30.309392 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee5e00-1832-4369-b436-56bbaef13e9a None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:30.309912 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee5e00-1832-4369-b436-56bbaef13e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:30.359763 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0bee5e00-1832-4369-b436-56bbaef13e9a None None] MFA Rules not processed for user `8731c4e543814bcf80075cd315a7b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:30.361252 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0bee5e00-1832-4369-b436-56bbaef13e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:30.458348 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1401/2866] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:30.462644 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de64807-09c7-4ace-8e8c-9ca62ff1deb3 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:30.483756 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de64807-09c7-4ace-8e8c-9ca62ff1deb3 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:30.488768 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1887-4811-4c01-bb7a-f3a4c77b5acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:30.489443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1887-4811-4c01-bb7a-f3a4c77b5acf admin admin] REQUEST_METHOD: `POST` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:30.489734 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1887-4811-4c01-bb7a-f3a4c77b5acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:30.490021 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d1887-4811-4c01-bb7a-f3a4c77b5acf admin admin] PATH_INFO: `/v3/projects` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:30.490729 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d1887-4811-4c01-bb7a-f3a4c77b5acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:30.492818 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d1887-4811-4c01-bb7a-f3a4c77b5acf admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:30.497744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:30.555748 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1466/2867] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 23 12:04:30 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:30.557064 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee5e00-1832-4369-b436-56bbaef13e9a None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:30.574412 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee5e00-1832-4369-b436-56bbaef13e9a None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:30.673742 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d20ed5-faf3-4132-864c-14519595ef58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:30.674984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d20ed5-faf3-4132-864c-14519595ef58 admin admin] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:30.675272 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d20ed5-faf3-4132-864c-14519595ef58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:30.675550 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d20ed5-faf3-4132-864c-14519595ef58 admin admin] PATH_INFO: `/v3/users` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:30.676258 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d20ed5-faf3-4132-864c-14519595ef58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:30.678514 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d20ed5-faf3-4132-864c-14519595ef58 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:30.709300 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1402/2868] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 23 12:04:30 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 12:04:30.713113 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1887-4811-4c01-bb7a-f3a4c77b5acf admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:30.730797 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d1887-4811-4c01-bb7a-f3a4c77b5acf admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:30.738280 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a01f9-b172-4762-ab8f-dc2dff438e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:30.739438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a01f9-b172-4762-ab8f-dc2dff438e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:30.740366 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a01f9-b172-4762-ab8f-dc2dff438e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:30.741203 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a01f9-b172-4762-ab8f-dc2dff438e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:30.743271 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a01f9-b172-4762-ab8f-dc2dff438e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:30.745899 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a01f9-b172-4762-ab8f-dc2dff438e8c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:30.760102 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1467/2869] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:04:30.764005 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d20ed5-faf3-4132-864c-14519595ef58 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:30.776496 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d20ed5-faf3-4132-864c-14519595ef58 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:30.781866 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437c6a8-3c4f-4602-8bd3-0c4b8a187923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:30.783278 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437c6a8-3c4f-4602-8bd3-0c4b8a187923 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:30.783615 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437c6a8-3c4f-4602-8bd3-0c4b8a187923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:30.784398 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9437c6a8-3c4f-4602-8bd3-0c4b8a187923 admin admin] PATH_INFO: `/v3/projects/2e4e826f92f04bd5a4ac850f8194ba78/users/01785777380f4a7dad5c03c7632572e5/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:30.799165 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:30.804225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9437c6a8-3c4f-4602-8bd3-0c4b8a187923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e4e826f92f04bd5a4ac850f8194ba78, user_id=01785777380f4a7dad5c03c7632572e5, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:30.808724 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9437c6a8-3c4f-4602-8bd3-0c4b8a187923 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:30.824921 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1403/2870] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:04:30 2022] PUT /identity/v3/projects/2e4e826f92f04bd5a4ac850f8194ba78/users/01785777380f4a7dad5c03c7632572e5/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:30.829025 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a01f9-b172-4762-ab8f-dc2dff438e8c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:30.842221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a01f9-b172-4762-ab8f-dc2dff438e8c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:30.936635 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0369d01-e9f5-4f9b-975e-795063de25b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:30.938177 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0369d01-e9f5-4f9b-975e-795063de25b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:30.938476 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0369d01-e9f5-4f9b-975e-795063de25b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:30.938760 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0369d01-e9f5-4f9b-975e-795063de25b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:30.940348 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0369d01-e9f5-4f9b-975e-795063de25b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:30.942994 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0369d01-e9f5-4f9b-975e-795063de25b9 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:30.965218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1468/2871] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 23 12:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 12:04:30.967253 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437c6a8-3c4f-4602-8bd3-0c4b8a187923 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:30.980347 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9437c6a8-3c4f-4602-8bd3-0c4b8a187923 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:30.987314 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda39fc-5cab-4d90-a9de-bade359a7b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:30.989220 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda39fc-5cab-4d90-a9de-bade359a7b04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:30.989864 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda39fc-5cab-4d90-a9de-bade359a7b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:30.995458 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda39fc-5cab-4d90-a9de-bade359a7b04 admin admin] PATH_INFO: `/v3/projects/2e4e826f92f04bd5a4ac850f8194ba78/users/01785777380f4a7dad5c03c7632572e5/roles/569707378794437b93d168fc8bb9cbc3` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:30.999408 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda39fc-5cab-4d90-a9de-bade359a7b04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e4e826f92f04bd5a4ac850f8194ba78, user_id=01785777380f4a7dad5c03c7632572e5, role_id=569707378794437b93d168fc8bb9cbc3)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:31.003241 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda39fc-5cab-4d90-a9de-bade359a7b04 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:31.043090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1404/2872] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 23 12:04:30 2022] PUT /identity/v3/projects/2e4e826f92f04bd5a4ac850f8194ba78/users/01785777380f4a7dad5c03c7632572e5/roles/569707378794437b93d168fc8bb9cbc3 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:34.438345 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0369d01-e9f5-4f9b-975e-795063de25b9 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:34.440788 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-e0369d01-e9f5-4f9b-975e-795063de25b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:34.462438 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0369d01-e9f5-4f9b-975e-795063de25b9 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:34.584705 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:34.585615 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:34.585998 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:34.586285 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] PATH_INFO: `/v3/users/62766b6eeb3d47beb5e3d056fb6d5d81` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:34.589494 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62766b6eeb3d47beb5e3d056fb6d5d81)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:34.591446 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:34.695575 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62766b6eeb3d47beb5e3d056fb6d5d81'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:34.696277 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62766b6eeb3d47beb5e3d056fb6d5d81'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:34.696933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62766b6eeb3d47beb5e3d056fb6d5d81'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:34.727641 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62766b6eeb3d47beb5e3d056fb6d5d81'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:34.742771 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:34.745526 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62766b6eeb3d47beb5e3d056fb6d5d81'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:34.766658 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1469/2873] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:04:34 2022] DELETE /identity/v3/users/62766b6eeb3d47beb5e3d056fb6d5d81 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:35.145055 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda39fc-5cab-4d90-a9de-bade359a7b04 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:35.147542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-7fda39fc-5cab-4d90-a9de-bade359a7b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:35.169404 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda39fc-5cab-4d90-a9de-bade359a7b04 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:35.254098 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:35.255029 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:35.255320 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:35.255601 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] PATH_INFO: `/v3/projects/a2d3e44f4113405a9cd4e604aea9955b` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:35.256938 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2d3e44f4113405a9cd4e604aea9955b)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:35.259095 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:35.343342 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2d3e44f4113405a9cd4e604aea9955b', 'request_id': 'req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:35.357018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2d3e44f4113405a9cd4e604aea9955b', 'request_id': 'req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:35.374014 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2d3e44f4113405a9cd4e604aea9955b', 'request_id': 'req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:35.423841 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] The token cache is being invalidate because project a2d3e44f4113405a9cd4e604aea9955b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:04:35.424150 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:35.425239 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:35.435018 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1405/2874] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:04:35 2022] DELETE /identity/v3/projects/a2d3e44f4113405a9cd4e604aea9955b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:35.438581 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:35.462317 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913db6d3-3f66-4f2d-9c30-5328fc45b882 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:35.688192 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:35.689327 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:35.689750 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:35.690132 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] PATH_INFO: `/v3/users/a5ba399ef99d421cb9870f4932d7379a` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:35.693105 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ba399ef99d421cb9870f4932d7379a)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:35.695486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:35.819265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ba399ef99d421cb9870f4932d7379a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:35.821184 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ba399ef99d421cb9870f4932d7379a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:35.826239 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ba399ef99d421cb9870f4932d7379a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:35.855031 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ba399ef99d421cb9870f4932d7379a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:35.868210 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ba399ef99d421cb9870f4932d7379a'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:35.910236 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1470/2875] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:04:35 2022] DELETE /identity/v3/users/a5ba399ef99d421cb9870f4932d7379a => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:35.923130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:35.925468 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:36.039830 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b12b0b-6794-4d1d-bc53-3bb7129dd6e0 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:36.067891 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:36.123262 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a22aca-0920-4bdc-ad8b-998a247aec4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:36.124560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a22aca-0920-4bdc-ad8b-998a247aec4d service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:36.124893 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a22aca-0920-4bdc-ad8b-998a247aec4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:36.125176 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a22aca-0920-4bdc-ad8b-998a247aec4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:36.128980 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-60a22aca-0920-4bdc-ad8b-998a247aec4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:36.248204 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a22aca-0920-4bdc-ad8b-998a247aec4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:36.249713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a22aca-0920-4bdc-ad8b-998a247aec4d service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:36.268542 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1406/2876] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:04:36.617420 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:36.620133 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:36.649179 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeaee9-1d93-478a-92e7-eaf370a8157a admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:36.660418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70930336-6680-42fe-8c76-6c401d3a2fe5 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:36.660418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70930336-6680-42fe-8c76-6c401d3a2fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:36.660418 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70930336-6680-42fe-8c76-6c401d3a2fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:36.669715 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:36.670781 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:36.671286 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:36.671792 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] PATH_INFO: `/v3/projects/971c124aa61549df80a67e641298d55f` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:36.677288 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=971c124aa61549df80a67e641298d55f)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:36.679268 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:36.732618 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-70930336-6680-42fe-8c76-6c401d3a2fe5 None None] MFA Rules not processed for user `01785777380f4a7dad5c03c7632572e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:36.734145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-70930336-6680-42fe-8c76-6c401d3a2fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:36.780449 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '971c124aa61549df80a67e641298d55f', 'request_id': 'req-4e073774-884b-425b-baef-2392990c11b8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:36.818218 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '971c124aa61549df80a67e641298d55f', 'request_id': 'req-4e073774-884b-425b-baef-2392990c11b8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:36.834253 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=79,keystone:INSERT=3,keystone:DELETE=18 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:36.837754 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '971c124aa61549df80a67e641298d55f', 'request_id': 'req-4e073774-884b-425b-baef-2392990c11b8'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:36.858155 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] The token cache is being invalidate because project 971c124aa61549df80a67e641298d55f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80243) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:04:36.858451 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:36.859297 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:04:36.869530 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1471/2877] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:04:36 2022] DELETE /identity/v3/projects/971c124aa61549df80a67e641298d55f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:04:36.940937 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:36.942749 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:36.957345 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1407/2878] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:37.060204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e073774-884b-425b-baef-2392990c11b8 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:37.068774 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac3dde-ffba-4adc-b86f-81701417407e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:37.074486 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac3dde-ffba-4adc-b86f-81701417407e service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:37.075100 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac3dde-ffba-4adc-b86f-81701417407e service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:37.075502 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac3dde-ffba-4adc-b86f-81701417407e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:37.078418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0dac3dde-ffba-4adc-b86f-81701417407e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:37.157915 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac3dde-ffba-4adc-b86f-81701417407e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:37.159861 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac3dde-ffba-4adc-b86f-81701417407e service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:37.180822 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1472/2879] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:04:46.921138 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=73 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:47.167205 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=60 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:04:53.479643 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd50779-c5d9-468c-9031-7393a322b521 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:53.480142 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd50779-c5d9-468c-9031-7393a322b521 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:53.480260 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd50779-c5d9-468c-9031-7393a322b521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:53.497343 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-bfd50779-c5d9-468c-9031-7393a322b521 None None] MFA Rules not processed for user `8731c4e543814bcf80075cd315a7b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:04:53.498557 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bfd50779-c5d9-468c-9031-7393a322b521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:53.505447 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1408/2880] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:04:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:04:53.545886 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac3dde-ffba-4adc-b86f-81701417407e service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:53.547306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-0dac3dde-ffba-4adc-b86f-81701417407e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:53.603489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac3dde-ffba-4adc-b86f-81701417407e service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:53.607208 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102d098-1b50-4124-a70f-f2c056d2762a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abcbb404d9a148568a82697abd51ec30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:53.607881 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102d098-1b50-4124-a70f-f2c056d2762a service cinder] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:53.608167 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102d098-1b50-4124-a70f-f2c056d2762a service cinder] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:53.608443 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102d098-1b50-4124-a70f-f2c056d2762a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:53.610362 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-6102d098-1b50-4124-a70f-f2c056d2762a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:53.624150 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102d098-1b50-4124-a70f-f2c056d2762a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:53.625601 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102d098-1b50-4124-a70f-f2c056d2762a service cinder] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:53.636584 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1473/2881] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:04:54.359714 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd50779-c5d9-468c-9031-7393a322b521 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:04:54.361153 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-bfd50779-c5d9-468c-9031-7393a322b521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:54.420649 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd50779-c5d9-468c-9031-7393a322b521 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:04:54.424237 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35286ec-f156-4400-8d4e-02406178994a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:04:54.424986 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35286ec-f156-4400-8d4e-02406178994a service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:04:54.425177 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35286ec-f156-4400-8d4e-02406178994a service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:04:54.425457 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35286ec-f156-4400-8d4e-02406178994a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:04:54.428023 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a35286ec-f156-4400-8d4e-02406178994a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:04:54.444113 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35286ec-f156-4400-8d4e-02406178994a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:04:54.445770 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35286ec-f156-4400-8d4e-02406178994a service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:04:54.457554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1409/2882] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:04:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:05:03.631580 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=26 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:05:04.452338 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=29 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:05:06.827328 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102d098-1b50-4124-a70f-f2c056d2762a service cinder] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:05:06.836831 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102d098-1b50-4124-a70f-f2c056d2762a service cinder] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:05:06.840429 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881d47b-88c0-47c9-9cf0-aba5ba9beb9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:05:06.841095 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881d47b-88c0-47c9-9cf0-aba5ba9beb9e service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:05:06.841377 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881d47b-88c0-47c9-9cf0-aba5ba9beb9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:05:06.841649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881d47b-88c0-47c9-9cf0-aba5ba9beb9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:05:06.843412 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4881d47b-88c0-47c9-9cf0-aba5ba9beb9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:05:06.853680 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881d47b-88c0-47c9-9cf0-aba5ba9beb9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:05:06.855085 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881d47b-88c0-47c9-9cf0-aba5ba9beb9e service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:05:06.863016 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1474/2883] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:05:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:05:08.792130 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35286ec-f156-4400-8d4e-02406178994a service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:05:08.793666 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-a35286ec-f156-4400-8d4e-02406178994a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:05:08.804641 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35286ec-f156-4400-8d4e-02406178994a service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:05:08.808546 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329bc1a-b363-48d5-8a44-a6ce9e32133a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:05:08.809227 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329bc1a-b363-48d5-8a44-a6ce9e32133a service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:05:08.809520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329bc1a-b363-48d5-8a44-a6ce9e32133a service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:05:08.809806 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329bc1a-b363-48d5-8a44-a6ce9e32133a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:05:08.871787 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329bc1a-b363-48d5-8a44-a6ce9e32133a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:05:08.874090 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329bc1a-b363-48d5-8a44-a6ce9e32133a service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:05:08.883851 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1410/2884] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:05:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:05:16.859509 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=6 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:05:18.878843 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=30 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:07:10.663731 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881d47b-88c0-47c9-9cf0-aba5ba9beb9e service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:07:10.666489 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-4881d47b-88c0-47c9-9cf0-aba5ba9beb9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:07:10.767469 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881d47b-88c0-47c9-9cf0-aba5ba9beb9e service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:07:10.776616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49342cf1-37ec-49c7-b3dd-4b445a3508a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:07:10.777315 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49342cf1-37ec-49c7-b3dd-4b445a3508a0 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:07:10.777911 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49342cf1-37ec-49c7-b3dd-4b445a3508a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:07:10.778527 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49342cf1-37ec-49c7-b3dd-4b445a3508a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:07:10.784442 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-49342cf1-37ec-49c7-b3dd-4b445a3508a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:07:10.800817 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49342cf1-37ec-49c7-b3dd-4b445a3508a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:07:10.802701 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49342cf1-37ec-49c7-b3dd-4b445a3508a0 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:07:10.814906 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1475/2885] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:07:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:07:11.551765 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d95a6-e9e1-4254-ab5e-dd8e9a00b2e6 None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:07:11.552289 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d95a6-e9e1-4254-ab5e-dd8e9a00b2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:07:11.552389 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d95a6-e9e1-4254-ab5e-dd8e9a00b2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:07:11.570944 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-0a3d95a6-e9e1-4254-ab5e-dd8e9a00b2e6 None None] MFA Rules not processed for user `8731c4e543814bcf80075cd315a7b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:07:11.572203 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0a3d95a6-e9e1-4254-ab5e-dd8e9a00b2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:07:11.579665 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1411/2886] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:07:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 12:07:11.904597 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49342cf1-37ec-49c7-b3dd-4b445a3508a0 service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:07:11.924419 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49342cf1-37ec-49c7-b3dd-4b445a3508a0 service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:07:11.928350 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df047c-22f4-4418-b14f-a114351772f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:07:11.929017 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df047c-22f4-4418-b14f-a114351772f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:07:11.929310 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df047c-22f4-4418-b14f-a114351772f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:07:11.929593 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df047c-22f4-4418-b14f-a114351772f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:07:11.931450 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-f1df047c-22f4-4418-b14f-a114351772f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:07:11.953337 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df047c-22f4-4418-b14f-a114351772f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:07:11.954744 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df047c-22f4-4418-b14f-a114351772f7 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:07:11.963265 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1476/2887] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:07:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:07:12.418356 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d95a6-e9e1-4254-ab5e-dd8e9a00b2e6 None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:07:12.419788 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-0a3d95a6-e9e1-4254-ab5e-dd8e9a00b2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:07:12.491599 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d95a6-e9e1-4254-ab5e-dd8e9a00b2e6 None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:07:12.497166 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5121d-6990-4c63-b1c9-6a9903309834 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:07:12.497816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5121d-6990-4c63-b1c9-6a9903309834 service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:07:12.498163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5121d-6990-4c63-b1c9-6a9903309834 service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:07:12.498380 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed5121d-6990-4c63-b1c9-6a9903309834 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:07:12.500139 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-8ed5121d-6990-4c63-b1c9-6a9903309834 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:07:12.512120 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5121d-6990-4c63-b1c9-6a9903309834 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:07:12.513713 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed5121d-6990-4c63-b1c9-6a9903309834 service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:07:12.523507 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1412/2888] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:07:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:07:21.959204 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=43 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:07:22.519275 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=35 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:07:25.167623 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df047c-22f4-4418-b14f-a114351772f7 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:07:25.183439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df047c-22f4-4418-b14f-a114351772f7 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:07:25.189144 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5088e-910e-476e-9ab0-aa47c73c2108 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:07:25.189795 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5088e-910e-476e-9ab0-aa47c73c2108 service nova] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:07:25.190207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5088e-910e-476e-9ab0-aa47c73c2108 service nova] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:07:25.190522 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5088e-910e-476e-9ab0-aa47c73c2108 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:07:25.192363 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-b8c5088e-910e-476e-9ab0-aa47c73c2108 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:07:25.205046 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5088e-910e-476e-9ab0-aa47c73c2108 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:07:25.206637 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c5088e-910e-476e-9ab0-aa47c73c2108 service nova] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:07:25.218207 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1477/2889] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:07:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:07:28.586041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5121d-6990-4c63-b1c9-6a9903309834 service placement] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:07:28.599694 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed5121d-6990-4c63-b1c9-6a9903309834 service placement] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:07:28.605916 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5cf191-9f1a-47bd-8fcc-af304653aecb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36c18bbd22f74efbae7aab1cdd3f6403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:07:28.606561 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cf191-9f1a-47bd-8fcc-af304653aecb service nova] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:07:28.606848 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cf191-9f1a-47bd-8fcc-af304653aecb service nova] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:07:28.607128 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5cf191-9f1a-47bd-8fcc-af304653aecb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:07:28.608926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-5a5cf191-9f1a-47bd-8fcc-af304653aecb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:07:28.622520 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5cf191-9f1a-47bd-8fcc-af304653aecb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:07:28.624163 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5cf191-9f1a-47bd-8fcc-af304653aecb service nova] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:07:28.635571 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1413/2890] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:07:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:07:35.213550 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=6 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:07:38.630400 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=6 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:07:56.196092 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5088e-910e-476e-9ab0-aa47c73c2108 service nova] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:07:56.216424 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5088e-910e-476e-9ab0-aa47c73c2108 service nova] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:07:56.221518 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c3408-c01e-460b-b370-b2e63e49531d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:07:56.222153 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c3408-c01e-460b-b370-b2e63e49531d service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:07:56.222439 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c3408-c01e-460b-b370-b2e63e49531d service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:07:56.222717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c3408-c01e-460b-b370-b2e63e49531d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:07:56.224525 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-653c3408-c01e-460b-b370-b2e63e49531d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:07:56.249931 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c3408-c01e-460b-b370-b2e63e49531d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:07:56.251516 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c3408-c01e-460b-b370-b2e63e49531d service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:07:56.261578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1478/2891] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:07:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:08:06.257248 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=15 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:09:05.896926 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a6e84-422b-482c-a00f-c27809de077b None None] REQUEST_METHOD: `POST` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:09:05.897490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a6e84-422b-482c-a00f-c27809de077b None None] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:09:05.897490 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a6e84-422b-482c-a00f-c27809de077b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:09:05.915816 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.auth.core [None req-000a6e84-422b-482c-a00f-c27809de077b None None] MFA Rules not processed for user `8731c4e543814bcf80075cd315a7b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80244) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 12:09:05.917061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-000a6e84-422b-482c-a00f-c27809de077b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:09:05.924061 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1414/2892] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 23 12:09:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 12:09:15.910984 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=3 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:10:54.856604 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c3408-c01e-460b-b370-b2e63e49531d service placement] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:10:54.873119 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c3408-c01e-460b-b370-b2e63e49531d service placement] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:10:54.876717 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75822c49-24bf-4f0f-ad47-552a26b0909e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:10:54.877361 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75822c49-24bf-4f0f-ad47-552a26b0909e service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:10:54.877649 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75822c49-24bf-4f0f-ad47-552a26b0909e service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:10:54.877930 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75822c49-24bf-4f0f-ad47-552a26b0909e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:10:54.885741 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75822c49-24bf-4f0f-ad47-552a26b0909e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:10:54.887175 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75822c49-24bf-4f0f-ad47-552a26b0909e service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:10:54.895152 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1479/2893] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:10:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:10:56.450567 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a6e84-422b-482c-a00f-c27809de077b None None] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:10:56.460277 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a6e84-422b-482c-a00f-c27809de077b None None] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:10:56.463873 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8adec-0f42-439a-986c-c2010184583b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:10:56.464504 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8adec-0f42-439a-986c-c2010184583b service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:10:56.464833 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8adec-0f42-439a-986c-c2010184583b service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:10:56.465117 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8adec-0f42-439a-986c-c2010184583b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:10:56.473244 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8adec-0f42-439a-986c-c2010184583b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:10:56.474663 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8adec-0f42-439a-986c-c2010184583b service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:10:56.483826 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1415/2894] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:10:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:10:57.497593 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75822c49-24bf-4f0f-ad47-552a26b0909e service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:10:57.506790 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75822c49-24bf-4f0f-ad47-552a26b0909e service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:10:57.510805 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13cb86-edf6-4ddc-b6da-f823ae566fa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:10:57.511290 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13cb86-edf6-4ddc-b6da-f823ae566fa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:10:57.511587 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13cb86-edf6-4ddc-b6da-f823ae566fa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:10:57.511869 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13cb86-edf6-4ddc-b6da-f823ae566fa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:10:57.532709 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13cb86-edf6-4ddc-b6da-f823ae566fa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:10:57.534224 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13cb86-edf6-4ddc-b6da-f823ae566fa8 service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:10:57.556980 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1480/2895] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:10:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 12:10:58.744914 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8adec-0f42-439a-986c-c2010184583b service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:10:58.754039 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8adec-0f42-439a-986c-c2010184583b service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:10:58.758483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6af52e-f38b-447f-b333-aeaf8fd0a10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:10:58.759128 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6af52e-f38b-447f-b333-aeaf8fd0a10b service neutron] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:10:58.759417 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6af52e-f38b-447f-b333-aeaf8fd0a10b service neutron] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:10:58.759697 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6af52e-f38b-447f-b333-aeaf8fd0a10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:10:58.770288 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6af52e-f38b-447f-b333-aeaf8fd0a10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:10:58.771708 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6af52e-f38b-447f-b333-aeaf8fd0a10b service neutron] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:10:58.783236 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1416/2896] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:10:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:11:07.554330 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=19 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:11:08.778323 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=12 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:11:09.648663 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13cb86-edf6-4ddc-b6da-f823ae566fa8 service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:11:09.657407 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13cb86-edf6-4ddc-b6da-f823ae566fa8 service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:11:09.661048 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a57a-09ff-4d21-a889-68357bead9ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f095d3370be457cb6687ac8f8c9f8ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:11:09.661693 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a57a-09ff-4d21-a889-68357bead9ef service neutron] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:11:09.661985 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a57a-09ff-4d21-a889-68357bead9ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:11:09.662288 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df61a57a-09ff-4d21-a889-68357bead9ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:11:09.664640 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-df61a57a-09ff-4d21-a889-68357bead9ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:11:09.749472 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a57a-09ff-4d21-a889-68357bead9ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:11:09.750900 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df61a57a-09ff-4d21-a889-68357bead9ef service neutron] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:11:09.769221 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1481/2897] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:11:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:11:12.774560 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6af52e-f38b-447f-b333-aeaf8fd0a10b service neutron] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:11:12.789145 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6af52e-f38b-447f-b333-aeaf8fd0a10b service neutron] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:11:12.794464 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8b1dc-bf30-42d1-86e1-3f7f8f69c313 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:11:12.795118 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8b1dc-bf30-42d1-86e1-3f7f8f69c313 service placement] REQUEST_METHOD: `GET` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:11:12.795443 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8b1dc-bf30-42d1-86e1-3f7f8f69c313 service placement] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:11:12.795683 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8b1dc-bf30-42d1-86e1-3f7f8f69c313 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:11:12.798224 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-39f8b1dc-bf30-42d1-86e1-3f7f8f69c313 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:11:12.812731 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8b1dc-bf30-42d1-86e1-3f7f8f69c313 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:11:12.814340 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8b1dc-bf30-42d1-86e1-3f7f8f69c313 service placement] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:11:12.825847 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1417/2898] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:11:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:11:15.710111 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a57a-09ff-4d21-a889-68357bead9ef service neutron] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:11:15.711616 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-df61a57a-09ff-4d21-a889-68357bead9ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:11:15.723893 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df61a57a-09ff-4d21-a889-68357bead9ef service neutron] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:11:15.730343 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:11:15.731351 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:11:15.731704 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:11:15.732050 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] PATH_INFO: `/v3/users/8731c4e543814bcf80075cd315a7b7ff` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:11:15.733353 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8731c4e543814bcf80075cd315a7b7ff)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:11:15.735338 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:11:15.786367 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8731c4e543814bcf80075cd315a7b7ff'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:15.787042 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8731c4e543814bcf80075cd315a7b7ff'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:15.788213 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8731c4e543814bcf80075cd315a7b7ff'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:15.800418 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8731c4e543814bcf80075cd315a7b7ff'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:15.808254 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8731c4e543814bcf80075cd315a7b7ff'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:15.822924 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1482/2899] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:11:15 2022] DELETE /identity/v3/users/8731c4e543814bcf80075cd315a7b7ff => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:11:15.999554 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8b1dc-bf30-42d1-86e1-3f7f8f69c313 service placement] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:11:16.015222 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8b1dc-bf30-42d1-86e1-3f7f8f69c313 service placement] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:11:16.074041 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:11:16.074899 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:11:16.075187 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:11:16.075493 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] PATH_INFO: `/v3/projects/c41820d2b5eb4dedb313bcef88259676` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:11:16.076721 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c41820d2b5eb4dedb313bcef88259676)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:11:16.078717 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:11:16.122888 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c41820d2b5eb4dedb313bcef88259676', 'request_id': 'req-ace91276-e4ee-428a-936f-6c0ca2cabd10'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.129335 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c41820d2b5eb4dedb313bcef88259676', 'request_id': 'req-ace91276-e4ee-428a-936f-6c0ca2cabd10'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.135355 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c41820d2b5eb4dedb313bcef88259676', 'request_id': 'req-ace91276-e4ee-428a-936f-6c0ca2cabd10'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.145196 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] The token cache is being invalidate because project c41820d2b5eb4dedb313bcef88259676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:11:16.145483 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.146105 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.148170 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1418/2900] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:11:15 2022] DELETE /identity/v3/projects/c41820d2b5eb4dedb313bcef88259676 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:11:16.152196 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:11:16.153710 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:11:16.167186 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b870d-2431-4dd9-a48f-f34851fc939c admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:11:16.200586 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:11:16.226421 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:11:16.227278 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:11:16.227568 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:11:16.227849 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] PATH_INFO: `/v3/users/01785777380f4a7dad5c03c7632572e5` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:11:16.229103 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01785777380f4a7dad5c03c7632572e5)` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:11:16.231007 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:11:16.276880 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01785777380f4a7dad5c03c7632572e5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.277531 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01785777380f4a7dad5c03c7632572e5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.278139 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01785777380f4a7dad5c03c7632572e5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.290887 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01785777380f4a7dad5c03c7632572e5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.297672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.notifications [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01785777380f4a7dad5c03c7632572e5'} {{(pid=80243) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.310238 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1483/2901] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 23 12:11:16 2022] DELETE /identity/v3/users/01785777380f4a7dad5c03c7632572e5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 12:11:16.482225 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] Authenticating user token {{(pid=80244) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:11:16.484302 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.fernet_utils [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80244) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:11:16.496753 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace91276-e4ee-428a-936f-6c0ca2cabd10 admin admin] Validating token access rules against request {{(pid=80244) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:11:16.551988 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794fb32-f806-4c52-b2fa-313122a8875a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '86b831b5471b48e6bf2bf9ce771279b2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e503e3101124c66a9726948b602dc87', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80244) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:11:16.552878 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794fb32-f806-4c52-b2fa-313122a8875a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:11:16.553596 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794fb32-f806-4c52-b2fa-313122a8875a admin admin] SCRIPT_NAME: `/identity` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:11:16.553888 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794fb32-f806-4c52-b2fa-313122a8875a admin admin] PATH_INFO: `/v3/projects/2e4e826f92f04bd5a4ac850f8194ba78` {{(pid=80244) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:11:16.555100 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794fb32-f806-4c52-b2fa-313122a8875a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e4e826f92f04bd5a4ac850f8194ba78)` {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:11:16.557092 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794fb32-f806-4c52-b2fa-313122a8875a admin admin] RBAC: Authorization granted {{(pid=80244) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:11:16.577838 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:11:16.603226 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2794fb32-f806-4c52-b2fa-313122a8875a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e4e826f92f04bd5a4ac850f8194ba78', 'request_id': 'req-2794fb32-f806-4c52-b2fa-313122a8875a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.610657 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2794fb32-f806-4c52-b2fa-313122a8875a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e4e826f92f04bd5a4ac850f8194ba78', 'request_id': 'req-2794fb32-f806-4c52-b2fa-313122a8875a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.617119 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2794fb32-f806-4c52-b2fa-313122a8875a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e4e826f92f04bd5a4ac850f8194ba78', 'request_id': 'req-2794fb32-f806-4c52-b2fa-313122a8875a'} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.628101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2794fb32-f806-4c52-b2fa-313122a8875a admin admin] The token cache is being invalidate because project 2e4e826f92f04bd5a4ac850f8194ba78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80244) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 12:11:16.628101 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2794fb32-f806-4c52-b2fa-313122a8875a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.628808 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG keystone.notifications [None req-2794fb32-f806-4c52-b2fa-313122a8875a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80244) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 12:11:16.630361 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: [pid: 80244|app: 0|req: 1419/2902] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 23 12:11:16 2022] DELETE /identity/v3/projects/2e4e826f92f04bd5a4ac850f8194ba78 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 12:11:26.306799 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:11:26.624562 dv-cinder-826608-3-iscsi devstack@keystone.service[80244]: DEBUG dbcounter [-] [80244] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80244) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 12:13:00.107852 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] Authenticating user token {{(pid=80243) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 12:13:00.109306 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:13:00.177672 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33057bf-790c-4cf6-a247-dba075a4d500 admin admin] Validating token access rules against request {{(pid=80243) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 12:13:00.238933 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea455507-5f3c-4805-85df-14a7f95b85ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee971347d23040d3984f961025e8157e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19c2f661cc934aa7869e9b3d4efa875e', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80243) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 12:13:00.239578 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea455507-5f3c-4805-85df-14a7f95b85ad service placement] REQUEST_METHOD: `GET` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 12:13:00.239863 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea455507-5f3c-4805-85df-14a7f95b85ad service placement] SCRIPT_NAME: `/identity` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 12:13:00.240141 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea455507-5f3c-4805-85df-14a7f95b85ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80243) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 12:13:00.241919 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.fernet_utils [None req-ea455507-5f3c-4805-85df-14a7f95b85ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80243) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 12:13:00.253872 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea455507-5f3c-4805-85df-14a7f95b85ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 12:13:00.255436 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea455507-5f3c-4805-85df-14a7f95b85ad service placement] RBAC: Authorization granted {{(pid=80243) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 12:13:00.265853 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: [pid: 80243|app: 0|req: 1484/2903] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 23 12:13:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 12:13:10.261626 dv-cinder-826608-3-iscsi devstack@keystone.service[80243]: DEBUG dbcounter [-] [80243] Writing DB stats keystone:SELECT=58 {{(pid=80243) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}